A Go package's tests pass individually under go test -run but fail when the whole package runs. How do you diagnose it?

seniorshould knowfreq 42%

answer

the failure is ordering, not one test
what does a test leave behind?
run the package twice in one process
randomise the order, keep the seed
let the testing package undo it

basics

~20 s

Treat it as order dependence, not flakiness: one test leaves process-global state behind, usually an environment variable or the working directory. Reproduce with go test -count=2 and -shuffle=on, then undo per test with t.Setenv and t.Chdir.

solid answer

~40 s

The signature - green in isolation, red as a suite - means one test is mutating state that outlives it inside the shared test binary. In this area that is almost always the environment (os.Setenv without a restore) or the working directory (os.Chdir), both process-wide. I reproduce deliberately rather than rerunning and hoping: go test -count=2 runs each test twice in one process and catches a test that poisons its own next run, and go test -shuffle=on randomises the order and prints a seed I can replay with -shuffle=<seed>. Bisecting with -run narrows it to a pair. The fix is t.Setenv and t.Chdir, which restore the previous state - including "it was unset" - through the test's cleanup; the deeper fix is passing configuration in as parameters so no test touches globals.

The signature - green in isolation, red as a suite - means one test is mutating state that outlives it inside the shared test binary. In this area that is almost always the environment (`os.Setenv` without a restore) or the working directory (`os.Chdir`), both process-wide. I reproduce deliberately rather than rerunning and hoping: `go test -count=2` runs each test twice in one process and catches a test that poisons its own next run, and `go test -shuffle=on` randomises the order and prints a seed I can replay with `-shuffle=<seed>`. Bisecting with `-run` narrows it to a pair. The fix is `t.Setenv` and `t.Chdir`, which restore the previous state - including "it was unset" - through the test's cleanup; the deeper fix is passing configuration in as parameters so no test touches globals.

code

go · 8 lines

func TestUsesRegion(t *testing.T) {
	t.Setenv("APP_REGION", "eu-west-1") // prior value, or unset, restored at cleanup
	t.Chdir(t.TempDir())               // working directory restored at cleanup

	if got := regionOrDefault(); got != "eu-west-1" {
		t.Fatalf("region = %q", got)
	}
}

go deeper

for a junior

Know that all the tests in one package run in a single process, and that t.Setenv and t.Chdir exist so a test can change the environment or the working directory without affecting the next one.

for a middle

Explain the mechanics of the restore: the previous value is recorded, unset counts as a value, and the undo runs through the test's cleanup so it happens even when the test fails.

for a senior

Show the diagnostic method rather than a guess — reproduce with -count=2 and -shuffle=on, keep the seed, bisect with -run, and then fix the leak at its source rather than pinning the order.

for a principal

Own the standard that keeps a large suite trustworthy: shuffled ordering enabled in CI, no process-global mutation in tests, and configuration read once at the edge so components are constructed directly in tests.

Step 1: reproduce it on purpose

Rerunning the suite and hoping is not diagnosis. Two flags turn the intermittent failure into a repeatable one:

go test -count=2 ./...
go test -shuffle=on ./...

-count=2 runs each test twice within the same process. It catches self-pollution — a test that passes on a clean environment and fails on the environment it left behind. This works even when the order never changes, which is why it finds bugs -shuffle misses.
-shuffle=on randomises the order of the top-level tests and prints the seed it used. That seed is the important part: go test -shuffle=<seed> replays exactly that order, turning "sometimes fails" into a command you can run in a loop while you fix it.

Then bisect with -run on the two suspects: run test A followed by test B and nothing else, and confirm B fails only after A.

Step 2: know what can leak

For this area of the standard library, the candidates are small and specific:

Environment variables. os.Setenv and os.Unsetenv change one table shared by the whole process. A test that sets APP_REGION and never restores it changes the meaning of every later test that reads it. Note that the os package serialises access to this table internally, so -race will not report it as a data race — the failure surfaces only as a wrong value, which is exactly why it is hard to find.
The working directory. os.Chdir moves it for the whole process, so a later test that opens a relative path looks somewhere unexpected. The classic version is a test that chdirs into a temporary directory that has since been deleted.
Package-level variables and anything initialised once, which is the same class of problem arriving from a different direction.

Step 3: fix it with the testing package, not by hand

The testing.T methods exist precisely for this:

func TestRegion(t *testing.T) {
    t.Setenv("APP_REGION", "eu-west-1")
    t.Chdir(t.TempDir())
    // ...
}

t.Setenv sets the variable and registers a cleanup that puts back whatever was there before, including the fact that it was previously unset. t.Chdir does the same for the working directory. Both unwind through the test's cleanup, so they run whether the test passes, fails, or calls t.Fatal. Neither may be used in a test that has been marked parallel, because the state they touch is shared.

Hand-rolled equivalents are where the bugs live. os.Setenv("K", "v") with defer os.Unsetenv("K") deletes a variable that may have had a real value in the developer's shell or in CI, so the leak simply moves to a different variable. And every new test has to remember to write the restore.

Step 4: remove the need

The strongest fix is that the code under test should not read process globals at all. Read the environment once, at the edge of the program in main, turn it into typed configuration, and pass it inward as parameters or struct fields. Then a test constructs the component directly with the values it wants, no global is touched, tests can run in parallel, and the whole class of ordering bug disappears. Similarly, a component that carries a base directory and joins names onto it never needs the process directory to be anything in particular.

What a strong answer sounds like

Name the symptom as order dependence; reach for -count=2 and -shuffle=on with the seed rather than rerunning; identify the process-global suspects specifically; fix with t.Setenv/t.Chdir; and finish with the design change that stops tests needing globals in the first place. A weak answer blames parallelism, adds a retry, or "fixes" it by pinning the test order — which preserves the bug and hides it until the day someone adds a test in the middle.

"Passes alone, fails together" is a precise symptom, not vague flakiness. It says the test binary carries state from one test into the next. All tests in a package run in a single process, so anything process-global is shared: the environment, the working directory, and package-level variables. ## Step 1: reproduce it on purpose Rerunning the suite and hoping is not diagnosis. Two flags turn the intermittent failure into a repeatable one: go test -count=2 ./... go test -shuffle=on ./... - **`-count=2`** runs each test twice within the same process. It catches self-pollution — a test that passes on a clean environment and fails on the environment it left behind. This works even when the order never changes, which is why it finds bugs `-shuffle` misses. - **`-shuffle=on`** randomises the order of the top-level tests and **prints the seed it used**. That seed is the important part: `go test -shuffle=<seed>` replays exactly that order, turning "sometimes fails" into a command you can run in a loop while you fix it. Then bisect with `-run` on the two suspects: run test A followed by test B and nothing else, and confirm B fails only after A. ## Step 2: know what can leak For this area of the standard library, the candidates are small and specific: - **Environment variables.** `os.Setenv` and `os.Unsetenv` change one table shared by the whole process. A test that sets `APP_REGION` and never restores it changes the meaning of every later test that reads it. Note that the `os` package serialises access to this table internally, so `-race` will *not* report it as a data race — the failure surfaces only as a wrong value, which is exactly why it is hard to find. - **The working directory.** `os.Chdir` moves it for the whole process, so a later test that opens a relative path looks somewhere unexpected. The classic version is a test that chdirs into a temporary directory that has since been deleted. - **Package-level variables and anything initialised once**, which is the same class of problem arriving from a different direction. ## Step 3: fix it with the testing package, not by hand The `testing.T` methods exist precisely for this: func TestRegion(t *testing.T) { t.Setenv("APP_REGION", "eu-west-1") t.Chdir(t.TempDir()) // ... } `t.Setenv` sets the variable and registers a cleanup that puts back **whatever was there before, including the fact that it was previously unset**. `t.Chdir` does the same for the working directory. Both unwind through the test's cleanup, so they run whether the test passes, fails, or calls `t.Fatal`. Neither may be used in a test that has been marked parallel, because the state they touch is shared. Hand-rolled equivalents are where the bugs live. `os.Setenv("K", "v")` with `defer os.Unsetenv("K")` **deletes** a variable that may have had a real value in the developer's shell or in CI, so the leak simply moves to a different variable. And every new test has to remember to write the restore. ## Step 4: remove the need The strongest fix is that the code under test should not read process globals at all. Read the environment once, at the edge of the program in `main`, turn it into typed configuration, and pass it inward as parameters or struct fields. Then a test constructs the component directly with the values it wants, no global is touched, tests can run in parallel, and the whole class of ordering bug disappears. Similarly, a component that carries a base directory and joins names onto it never needs the process directory to be anything in particular. ## What a strong answer sounds like Name the symptom as order dependence; reach for `-count=2` and `-shuffle=on` with the seed rather than rerunning; identify the process-global suspects specifically; fix with `t.Setenv`/`t.Chdir`; and finish with the design change that stops tests needing globals in the first place. A weak answer blames parallelism, adds a retry, or "fixes" it by pinning the test order — which preserves the bug and hides it until the day someone adds a test in the middle.

Why prefer t.Setenv over os.Setenv with a defer that unsets the variable?
`t.Setenv` records the previous state — including that the variable was unset — and restores it through the test's cleanup. A hand-rolled `defer os.Unsetenv` deletes a variable that may have held a real value in the developer's shell or in CI, so the leak just moves. It also has to be repeated correctly in every new test.
What does -count=2 catch that -shuffle=on does not?
Self-pollution: a test that passes against a clean environment and fails against the state it left behind on its own previous run. That reproduces even when the order never varies. `-shuffle=on` catches the other half — one test depending on state a *different* test happened to leave, which only shows up when the order changes.
Will the race detector find a leaked environment variable?
No. The `os` package serialises access to the environment internally, so concurrent reads and writes are not a data race and `-race` stays quiet. The symptom is a wrong value, not a race report. The detector will, however, help with the package-level variables that often accompany this kind of leak.
How do you make the ordering bug impossible rather than fixed?
Stop the code under test from reading process globals. Read the environment once in `main`, build typed configuration, and pass it in as parameters or struct fields; carry a base directory instead of relying on the process working directory. Tests then construct the component with the values they want and touch no shared state at all.

saying these in an interview costs you the question

Calls it flakiness and adds a retry
Pins the test order instead of removing the dependence
Uses os.Setenv in a test and never restores it
Expects the race detector to catch a leaked environment variable
Thinks -shuffle=on makes the failure impossible to reproduce

A Go package's tests pass individually under go test -run but fail when the whole package runs. How do you diagnose it?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

Step 1: reproduce it on purpose

Step 2: know what can leak

Step 3: fix it with the testing package, not by hand

Step 4: remove the need

What a strong answer sounds like

▾common follow-ups4 questions

▾red flags5 flags

Step 1: reproduce it on purpose

Step 2: know what can leak

Step 3: fix it with the testing package, not by hand

Step 4: remove the need

What a strong answer sounds like

answer

go deeper