skip to content

Why does re-running `go test` after a flaky failure print `(cached)`, and how do you force a real re-run?

level: juniorimportance: should knowfreq 55%

answer

  1. the second run never actually ran
  2. only green results are stored
  3. a restricted set of cacheable flags
  4. one flag turns it off, and it is not -race
  5. go clean names a different cache per flag

basics

~20 s

Go's test cache stores the result of a run that succeeded and replays it when nothing relevant changed, printing (cached) without executing anything. Add -count=1 to force a real run, or discard the stored results with go clean -testcache.

solid answer

~40 s

The go command caches successful test results. If the test binary, the command line (only a small set of flags counts as cacheable), the files the test opened and the environment variables it read are all unchanged, `go test` prints `ok pkg (cached)` and runs nothing at all. Failures are never cached, so the red run you saw was real — but the green re-runs you do while chasing a flake may be replays of an older pass, which makes the flake look like it went away. The idiomatic bypass is `go test -count=1 ./...`: `-count` is not one of the cacheable flags, so the binary really executes. `go clean -testcache` throws the stored results away. When you are hunting nondeterminism you should be passing `-count=N` anyway, which sidesteps the cache for free.

code

text · 8 lines
text
$ go test ./internal/scheduler
ok      example.com/svc/internal/scheduler      0.412s

$ go test ./internal/scheduler
ok      example.com/svc/internal/scheduler      (cached)

$ go test -count=1 ./internal/scheduler
ok      example.com/svc/internal/scheduler      0.407s

go deeper

for a junior

Be ready to say that a successful test run is cached and replayed as (cached), and that -count=1 is the standard way to force a real execution. Knowing that failures are never cached is the other half of the answer.

for a middle

An interviewer expects the mechanics: what goes into the cache key — the binary, a restricted set of cacheable flags, the files read and the environment variables consulted — and why -count is deliberately outside that set.

for a senior

Show the triage habit. Every command you type about a suspect test carries -count=N, so you never mistake a replay for evidence, and you can explain why a cold CI cache makes the flake visible there first.

for a principal

Own the policy angle: whether CI should preserve the test cache between runs at all, and the tradeoff between suite wall-clock time and getting an honest re-execution of every package on every merge.

## What the test cache is The `go` command keeps a **build cache** (under `GOCACHE`) and, inside it, a **test cache**: a record of test runs that *succeeded*, keyed by everything the go command believes could change the outcome. When you type `go test ./...` again and that key still matches, the go command prints the stored result with the marker `(cached)` in place of the elapsed time, and the test binary is never started. This is a build-system optimisation, not a testing feature. Its purpose is the same as caching a compiled package: in a repository with a hundred packages, re-running the ninety-nine you did not touch is pure waste. It is also the single most confusing thing about `go test` for someone chasing an intermittent failure, because the tool reports success for a run that did not happen. ## What is in the key A cached result is reused only when all of the following match the earlier run: - **The test binary itself** — which means the package's source, everything it imports, the compiler version and the build flags. Change a line anywhere in the dependency graph and the entry is gone. - **The command line**, which must consist *entirely* of flags the go command considers cacheable. That restricted set is small — `-benchtime`, `-cpu`, `-list`, `-parallel`, `-run`, `-short`, `-timeout`, `-failfast`, `-fullpath` and `-v`. Any other test flag, any non-test flag, or any extra argument makes the run uncacheable. - **The files the test opened** and **the environment variables it read** during the run. The go command observes these and stores them with the result; if the contents of a file the test read have changed, or a variable it consulted now has a different value, the entry is invalidated. This is what lets a table test that reads a fixture file be cached safely. Only **successful** runs are stored. A failing run is always a real run, and it is never replayed. ## Why it bites when you are chasing a flake The workflow that produces the confusion is: CI goes red, you pull the branch, you run `go test ./internal/scheduler`, it prints `ok ... 0.4s`, you shrug and re-run it four more times to be sure. Runs two through five print `(cached)` and take milliseconds. You conclude the failure was infrastructure. In fact you executed the test exactly once. The defence is to make every diagnostic run uncacheable on purpose: - `go test -count=1 ./internal/scheduler` — the documented idiom for "actually run it". `-count` is deliberately outside the cacheable set precisely so that this works. - `go test -count=200 -run TestWorkerPoolResults ./internal/scheduler` — what you want anyway when the failure is one run in fifty. Repetition both defeats the cache and multiplies your chances of seeing the bad interleaving. - `go clean -testcache` — expires *all* cached test results. Useful once, at the start of a session; reaching for it before every run is a sign you have not understood `-count=1`. Note the three `go clean` flags name three different caches, and mixing them up is common: `-testcache` expires stored test results, `-cache` deletes the whole build cache so everything recompiles, and `-modcache` deletes downloaded module source. Only the first is what you want here. ## What the cache is not It is not a per-test cache: the unit is a **package's** test run, not an individual `Test` function. It does not make a flaky test less flaky — it hides the flakiness by refusing to roll the dice again. And it is not the reason a test is fast: a genuinely cached run reports `(cached)` explicitly, so if you are unsure whether work happened, read the marker rather than the wall-clock number. ## The habit to build Treat `(cached)` as "no information". When a test is under suspicion, every command you type about it should carry `-count=N` — `-count=1` when you just want one honest run, a larger number when you are trying to reproduce. In CI it matters much less: a fresh container usually has a cold cache, which is part of why a flake shows up there and not on your machine, where half your re-runs never executed.

  • Can a failing test run ever be served from the cache?
    No. Only runs that succeeded are stored, so a failure is always a genuine execution. That asymmetry is worth remembering while triaging: the red result you are looking at happened, but the green results you collected afterwards may be replays of a single earlier pass.
  • Besides the source, what invalidates a cached test result?
    The command line — using any flag outside the cacheable set makes the run uncacheable — plus the files the test opened and the environment variables it read, both of which the go command records with the result. Change a fixture file's contents or a variable the test consults and the entry no longer matches.
  • How do go clean -testcache and go clean -cache differ?
    `go clean -testcache` expires only the stored test results, so the next `go test` re-runs the binaries but does not recompile the world. `go clean -cache` deletes the entire build cache, throwing away compiled packages as well, which makes the next build slow for no diagnostic benefit.

saying these in an interview costs you the question

  • Reads (cached) as proof the test just passed again
  • Thinks failing runs are cached too
  • Adds -race purely to defeat the cache instead of -count=1
  • Confuses go clean -testcache with -modcache or -cache
  • Believes editing an unrelated package re-runs this one