skip to content

How do you enable a Go runtime tracer such as GODEBUG=gctrace=1 without rebuilding the binary?

level: juniorimportance: should knowfreq 45%

answer

  1. not a flag, not a config file
  2. the launcher supplies it, the code does not
  3. one variable, comma-separated name=value pairs
  4. read at start-up, written to standard error

basics

~20 s

GODEBUG is an environment variable the Go runtime reads when the process starts, so you set it in whatever launches the binary: GODEBUG=gctrace=1 ./job. It holds a comma-separated list of name=value settings and needs no code change.

solid answer

~50 s

GODEBUG is a single environment variable holding a comma-separated list of `name=value` settings that the Go runtime, and a few standard-library packages, parse during process start-up. The program opts into nothing: you prefix the launch command or add the variable to the job definition that starts the process, and the same binary starts reporting. `gctrace=1` prints a line per garbage collection, `inittrace=1` a line per package that does init work, `schedtrace=1000` a scheduler summary every 1000 ms, and `net/http` reads `http2debug=1` for verbose HTTP/2 logging. All of it goes to standard error through the runtime's own low-level printer, not through `log` or `log/slog`, so your logging configuration cannot route it and you have to capture file descriptor 2 to keep it. The tracer settings are read once at start-up, so switching one on or off costs a restart.

code

text · 7 lines
text
# no rebuild, no flag, no code change
$ ./nightly-import -config /etc/import.toml

$ GODEBUG=inittrace=1,gctrace=1 ./nightly-import -config /etc/import.toml 2>godebug.log

# wrong: the second assignment replaces the first, so only inittrace is on
$ GODEBUG=gctrace=1 GODEBUG=inittrace=1 ./nightly-import

go deeper

for a junior

Be ready to name GODEBUG as an environment variable, write one setting correctly, and say plainly that the binary needs no rebuild and no code change.

for a middle

Explain that the runtime parses GODEBUG during start-up before user init, that several settings share one comma-separated variable, and that output goes to standard error rather than through the program's logger.

for a senior

Show how you get that output out of a real deployment: capturing file descriptor 2 in the job definition, accepting that enabling or disabling a tracer costs a restart, and keeping the setting scoped to a single run.

for a principal

Own the policy: which diagnostic environment variables a service may be relaunched with, who may set them, and why unstructured runtime output is a debugging aid rather than a telemetry contract other systems build on.

## What GODEBUG is `GODEBUG` is one environment variable whose value is a comma-separated list of `name=value` settings, for example `GODEBUG=gctrace=1,inittrace=1`. The Go runtime parses it very early in start-up, before any package-level initialisation and long before `main` runs, and copies the values into internal variables that the rest of the runtime consults. Nothing in the program has to import a package, register a handler or call a function for this to work: every Go binary responds to `GODEBUG`, including one somebody else built and shipped to you. That is the property that makes it valuable. A binary already deployed on a machine you cannot rebuild — a nightly batch job launched by a scheduler, a vendored tool, a container image you do not own the Dockerfile for — can still be asked to report on its own runtime, just by changing the environment it is launched with. ## The settings this family covers - `gctrace=1` — one line on standard error per garbage collection, summarising the work and the pause. - `inittrace=1` — one line per package that has init work, with when its init started, how long it took, and how much it allocated. - `schedtrace=N` — a one-line summary of Go's goroutine scheduler every N **milliseconds**. - `scheddetail=1` — turns each `schedtrace` tick into a multi-line dump of every P, M and goroutine. It does nothing on its own; `schedtrace` must also be set. - `asyncpreemptoff=1` — the odd one out: it does not report anything, it disables signal-based asynchronous goroutine preemption. It is a bisection tool, not a tracer. - `http2debug=1` (and `=2`, which adds frame dumps) — read by `net/http`, not the runtime, and a reminder that GODEBUG is a shared channel that several standard-library packages listen on. ## How you actually set it Because it is an environment variable, everything you already know about environment variables applies. You can prefix a single command: ``` GODEBUG=inittrace=1 ./nightly-import ``` You can put it in the job, unit or task definition your scheduler uses, and a child process inherits it from its parent unless the parent scrubs the environment. Two mistakes are common. The first is writing two assignments — `GODEBUG=gctrace=1 GODEBUG=inittrace=1 ./job` — where the second simply replaces the first, so only one setting is live; the settings share one variable and are separated by commas. The second is a wrapper script or a launcher that sanitises the environment and drops `GODEBUG` before `exec`, so the variable never reaches the process that matters. ## Where the output goes Runtime tracer output is written to **standard error** by the runtime's own printing routine. It is not a `log` or `log/slog` record: there are no timestamps, no severity, no JSON, no fields, and no way to redirect it from inside the program's logging setup. It interleaves, unsynchronised, with anything else the process writes to stderr. If your job runner captures only stdout into the job log, the lines are simply discarded, and you will conclude the setting did nothing. Redirecting `2>` to its own file at launch is usually the right move, so the diagnostic stream stays separate from the application log. The formats are documented as subject to change between releases. They are for a human reading a bounded diagnostic run, not a stable interface to parse. ## Read once, at start-up `gctrace`, `inittrace`, `schedtrace`, `scheddetail` and `asyncpreemptoff` are read into runtime variables during start-up. Calling `os.Setenv("GODEBUG", "gctrace=1")` from a running program — say, from an admin endpoint — does not turn them on; a small number of GODEBUG settings support run-time updates, but these tracers are not among them. Practically: enabling or disabling one is a restart, so you plan it into a scheduled run or a deliberate bounce rather than reaching for it mid-incident on a process you must not restart. ## Why this matters in an interview An engineer arriving from another ecosystem expects a command-line flag or a config file entry, and looks for `--gc-log` or a logging level. Go's answer is an environment variable read by the runtime, with unstructured output on stderr and no in-process control surface. Knowing that is the difference between saying "we cannot see anything without a new build" and getting an answer out of tonight's run.

  • Does the program have to import anything or call a runtime function for GODEBUG to take effect?
    No. The runtime parses GODEBUG itself during start-up, before package initialisation and before `main`, so any Go binary responds to it. That is exactly what makes it useful on a binary you did not build and cannot rebuild.
  • Where does GODEBUG output go, and can you route it into the application's structured logs?
    It goes to standard error, written by the runtime's own low-level printer: no timestamps, no levels, no fields, and no path through `log` or `log/slog`. You cannot redirect it from inside the program's logging setup — you capture file descriptor 2 at launch, and expect it to interleave with anything else the process writes to stderr.
  • Can you switch gctrace on in a process that is already running?
    No. `gctrace`, `inittrace`, `schedtrace`, `scheddetail` and `asyncpreemptoff` are read into runtime variables at start-up, and calling `os.Setenv("GODEBUG", ...)` later does not enable them. A few GODEBUG settings do support run-time updates, but the runtime tracers do not, so plan them into a restart.

It is closer to a kernel boot parameter than to an application log level: you set it on the way in, the process reads it once, and turning it off means starting the process again.

saying these in an interview costs you the question

  • Says you must rebuild the binary with a build tag to get tracing
  • Thinks GODEBUG is a flag passed to go build or go run
  • Expects the tracer lines to appear in the application's JSON logs
  • Writes two GODEBUG assignments instead of one comma-separated list
  • Believes os.Setenv can switch gctrace on mid-run