skip to content

Benchmarks and Fuzzing

4 roadmaps17 questions

Two artefacts the testing package runs but does not assert on: a benchmark that measures cost, and a fuzz target that hunts for a counterexample. Each brings one hazard of its own.

part ofGo (Golang)overview, primer and where to startread it →
on this pageshow

explore

questions

17

Where does `go test -fuzz` save an input that makes a fuzz target fail, and what replays it later?

level: juniorimportance: must knowfreq 42%

answer

  1. written into your source tree, not a cache
  2. package directory, under a testdata subfolder
  3. one file per failing input, named by hash
  4. plain go test replays it as a subtest
  5. committing it makes a permanent regression test

basics

~20 s

The fuzzing engine writes the failing input into a new file under testdata/fuzz/<FuzzName>/ in the package's own directory. That file becomes a seed corpus entry, so every later plain go test run replays it, with no -fuzz flag needed.

solid answer

~50 s

When `go test -fuzz=FuzzParseFrame ./proxy` finds an input that panics or fails the target, it minimizes the input and writes it to `proxy/testdata/fuzz/FuzzParseFrame/<hash>` in the package's source directory, then prints that path and a `-run` selector that replays just that entry. Files under `testdata/fuzz/<FuzzName>/` are the seed corpus: an ordinary `go test ./proxy` with no `-fuzz` flag runs each of them once, as a subtest named after the file. So the crasher becomes a permanent, deterministic regression test the moment you commit it. The file is small and textual - a `go test fuzz v1` header line followed by one line per fuzz argument written as `type(value)` - so it reviews like source rather than like a binary blob. Because it lands in your working tree and not in a cache, it survives only if you actually commit or archive it.

code

text · 2 lines
text
go test fuzz v1
[]byte("\x02\x00\x00\x00\xff")

go deeper

for a junior

Be ready to say that the failing input is written into testdata/fuzz under the package, and that committing that file is what turns a one-off crash into a test that runs forever.

for a middle

Explain the mechanics: the entry is a text file with a go test fuzz v1 header, it becomes a subtest named after the file, and a plain go test executes it with no fuzzing involved.

for a senior

Show that you treat the file as the output of the fuzz run - it has to be archived or committed before the workspace disappears, and it should land in the same change as the fix.

for a principal

Own the consequence: every committed entry runs on every build from then on, and in a public repository it publishes a working trigger for a bug your users may not have patched yet.

### What a fuzz failure actually produces A Go fuzz target is a test function the toolchain can call with generated inputs. Running `go test -fuzz=FuzzParseFrame ./proxy` on a package that parses a binary protocol frame will eventually feed it a byte sequence that makes it panic or report an error. Two things then happen, in this order. First the engine **minimizes** the input: it re-runs the target on smaller and simpler variants, keeping ones that still fail, so that what you get is a compact repro rather than the four kilobytes of random noise that happened to trip the parser. Second it **writes the minimized input into your source tree**, at `proxy/testdata/fuzz/FuzzParseFrame/<hash>`, where the directory is named after the fuzz function and the file is named by a hash of its contents. The failure output names the path it wrote, and also prints the `go test -run=...` command that replays that single entry. The important word is *source tree*. The file is created in the package directory of the working copy the test ran in - not in a cache, not in a temporary directory. On a laptop, `git status` shows it as a new untracked file the moment the run finishes. ### What is inside the file A corpus entry is plain text. The first line is a format marker, `go test fuzz v1`. Each following line is one argument to the fuzz function, written as a Go-style typed literal - `[]byte("...")`, `string("...")`, `int(7)`, `bool(true)` and so on. The number and order of those lines must match the target's argument list, which is why a corpus file for a two-argument target has two value lines. Because entries are text, they diff cleanly, review like source, and can be hand-written. If you know a frame with a length prefix larger than the remaining bytes should be rejected, you can add that case as a file yourself rather than waiting for the engine to stumble on it. ### Why `testdata/fuzz` is the magic path `testdata` is the conventional directory name that the `go` tool skips when it looks for packages, so anything under it is data rather than code. Inside it, `fuzz/<FuzzName>/` has a defined meaning to the testing package: every file in that directory is a **seed corpus entry** for that fuzz target. Seed entries are used in two distinct situations: - During a `-fuzz` run they are executed first, before any generated input, as the starting points for the coverage-guided search. - During an **ordinary** `go test` - no `-fuzz` flag anywhere, including whatever your CI already runs - the fuzz target still executes, once per seed entry, as a normal deterministic test. Each entry appears as a subtest named after its file. That second bullet is the whole payoff. A committed crasher is not "some fuzzing artefact"; it is a regression test that runs in milliseconds on every build, forever, without anyone needing to enable fuzzing. ### Replaying one entry Because each entry is a subtest, the standard `-run` selector picks it out: `go test -run='FuzzParseFrame/1f0a9c2e' ./proxy` No `-fuzz`, no randomness, no time budget - it feeds exactly that one saved input and reports pass or fail. This is the fastest possible loop while you are fixing the bug, and it is what the failure output tells you to run. ### If you never commit it The entry only exists in the working tree it was written into. A discarded container, a `git clean`, or a colleague reproducing on their own machine all leave you with nothing but a log line. The coverage-guided corpus that led the engine to the input is separate again and machine-local, so re-running fuzzing may take a very long time to rediscover the same case, or never do so. Treat the file as the deliverable of the fuzz run: commit it beside the fix, or at minimum archive it, before the machine that produced it goes away.

  • How do you re-run only that one saved crasher instead of the whole fuzz target?
    Each file in `testdata/fuzz/FuzzParseFrame` runs as a subtest named after the file, so `go test -run='FuzzParseFrame/1f0a9c2e' ./proxy` executes just that entry with no fuzzing at all. The failure output prints exactly that command when it saves the file. Since no `-fuzz` flag is involved it is fast, deterministic and works anywhere, including an ordinary CI test job.
  • What is actually inside one of those corpus files?
    Plain text. The first line is `go test fuzz v1`, identifying the format, and each following line is one argument written as a Go-style typed literal such as `[]byte("...")` or `int(7)`, in the same order as the target's parameters. Because it is text it diffs and reviews like source, and you can hand-write an entry to pin a case you care about.
  • If nobody commits the file, is the input lost?
    Effectively yes. It exists only in the working tree that produced it, so a discarded container or a `git clean` takes it away. The coverage-guided corpus that led the engine there lives in the build cache, which is machine-local too, so a fresh fuzzing run may need a very long time to rediscover the same input - or may never reach it.

saying these in an interview costs you the question

  • Says the crasher is stored in the module cache
  • Thinks a plain go test ignores testdata/fuzz entries
  • Believes the saved entry is an opaque binary blob
  • Assumes a later fuzzing run will rediscover it automatically
  • Expects to reproduce the bug from the log line alone
open as a page

What does a Go fuzz target `func FuzzParseQuery(f *testing.F)` do, and what are `f.Add` and `f.Fuzz` for?

level: juniorimportance: must knowfreq 48%

basics

~20 s

A Go fuzz target is a FuzzXxx function in a _test.go file that takes *testing.F. f.Add registers seed inputs; f.Fuzz takes the callback the engine runs, first on those seeds and then on mutated variations of them.

open as a page

In a Go benchmark func BenchmarkX(b *testing.B), what is b.N and who sets its value?

level: juniorimportance: must knowfreq 70%

basics

~20 s

b.N is the iteration count the testing package hands a benchmark; you never set it. go test calls the whole function repeatedly with a growing b.N until the run reaches -benchtime, then reports elapsed time divided by b.N.

open as a page

A Go benchmark that hashes a 1 MiB buffer reports 0.3 ns/op — what most likely happened?

level: juniorimportance: should knowfreq 42%

basics

~20 s

The compiler deleted the work. A function with no side effects whose result is never used becomes dead code once it is inlined, so the loop measures only its own counter. Keep the result alive by storing it in a package-level variable.

open as a page

How does the seed corpus under `testdata/fuzz` differ from the fuzzing corpus Go keeps in the build cache?

level: middleimportance: should knowfreq 34%

basics

~20 s

testdata/fuzz is checked-in source: it travels with the repository and runs on every plain go test. The generated corpus lives in the build cache directory, is machine-local, is only used while fuzzing, and go clean -fuzzcache deletes it.

open as a page

In a Go benchmark, why store the computed result in a package-level variable rather than a local one?

level: middleimportance: should knowfreq 34%

basics

~20 s

A store to a package-level variable is observable outside the function, so the compiler must perform it, which keeps the computation alive. A local nothing reads is a dead store; the blank identifier observes nothing.

open as a page

What does `go test -fuzz=FuzzParseQuery -fuzztime=30s` do that a plain `go test` does not?

level: middleimportance: should knowfreq 40%

basics

~20 s

It switches the package into fuzzing mode: after the ordinary tests and the seeds, the engine generates mutated inputs for 30 seconds, keeping any that reach new coverage. Plain go test only replays the seeds and stops.

open as a page

In go test -benchmem output, what do B/op and allocs/op measure, and what does b.ReportAllocs do?

level: middleimportance: should knowfreq 50%

basics

~20 s

B/op is the average bytes of heap memory allocated per iteration and allocs/op the average number of heap allocations. Pass -benchmem to go test for every benchmark, or call b.ReportAllocs inside one benchmark to always report those two columns.

open as a page

What do b.ResetTimer and b.StopTimer/b.StartTimer control in a Go benchmark?

level: middleimportance: should knowfreq 55%

basics

~10 s

b.ResetTimer zeroes the time and allocations counted so far, so fixture setup above the loop is not charged to ns/op. b.StopTimer and b.StartTimer pause and resume that same accounting around work inside the loop.

open as a page

An overnight `go test -fuzz` job crashed a frame parser, but nobody can reproduce it now. What went wrong?

level: seniorimportance: should knowfreq 30%

basics

~20 s

The failing input was written into the job's working tree under testdata/fuzz, and the coverage-guided corpus into that machine's build cache. A throwaway container destroys both, so only a log line survives and the crash is effectively gone.

open as a page

How do you prove a Go benchmark's fast ns/op comes from the compiler deleting the loop body rather than from real speed?

level: seniorimportance: should knowfreq 30%

basics

~10 s

Scale the input and see whether the timing moves, check the implied throughput against physical limits, and disassemble the built test binary. Then re-run with the result anchored: an orders-of-magnitude jump proves deletion.

open as a page

What should an f.Fuzz callback assert when fuzzing a query parser that reads untrusted input?

level: seniorimportance: should knowfreq 44%

basics

~10 s

Assert properties that hold for every input, not expected outputs: the parser never panics, an error return is an acceptable outcome, a successfully parsed query survives a round trip, and declared bounds are respected.

open as a page

When a Go fuzz run finds a crasher, how do you decide whether it gets committed under `testdata/fuzz`?

level: principalimportance: should knowfreq 24%

basics

~20 s

Default yes, but with conditions. Every committed entry runs on every go test forever, and in a public repository it publishes a working trigger. So commit minimized entries, one per distinct root cause, and sequence publication after the fix ships.

open as a page

What does `go test`'s `-fuzzminimizetime` flag control when a fuzz target hits a failing input?

level: middleimportance: nice to knowfreq 22%

basics

~20 s

It bounds how long the fuzzing engine spends shrinking a failing input before saving it. After a failure Go repeatedly retries smaller variants that still fail; -fuzzminimizetime caps each such minimization attempt, and the default is 60s.

open as a page

In a Go benchmark, what does a for b.Loop() loop do that a for i := 0; i < b.N; i++ loop does not?

level: middleimportance: nice to knowfreq 26%

basics

~10 s

It keeps the arguments and results of calls in the loop alive, so the compiler cannot delete the body, and it bounds the timed region itself, so setup above the loop is not measured.

open as a page

In Go fuzzing, which argument types may the f.Fuzz callback take, and how must f.Add match them?

level: middleimportance: nice to knowfreq 30%

basics

~20 s

After the leading *testing.T, only string, []byte, bool, float32, float64 and the signed and unsigned integer types are allowed - no structs, maps or other slices. Every f.Add call must pass exactly those types, in order.

open as a page

A Go benchmark reports different ns/op at -benchtime=1s and -benchtime=10s. What causes that?

level: seniorimportance: nice to knowfreq 32%

basics

~20 s

A longer benchmark duration means a larger b.N, and ns/op is the whole timed run divided by it. A figure that moves means iterations are not independent: one-off setup amortised away, or state accumulating across iterations.

open as a page