skip to content

What should an f.Fuzz callback assert when fuzzing a query parser that reads untrusted input?

level: seniorimportance: should knowfreq 44%

answer

  1. you cannot predict the output
  2. not crashing is already a property
  3. an error is a valid result
  4. there and back again
  5. invariants, never expected values

basics

~10 s

Assert properties that hold for every input, not expected outputs: the parser never panics, an error return is an acceptable outcome, a successfully parsed query survives a round trip, and declared bounds are respected.

solid answer

~50 s

You cannot predict the correct output for a mutated byte sequence, so the callback checks invariants, not values. The free property is that it must not panic — the fuzzing engine already fails on an index out of range, a nil dereference or a slice bounds error inside the parser, which is exactly the bug class a hostile client finds. On top of that: return early when the parser returns an error, because rejecting nonsense is correct behaviour and failing on it would end the search on the first mutation; assert a round trip, that re-parsing the printed form of a successfully parsed query yields the same structure; and assert declared bounds, such as the parsed limit never exceeding the caller's. Keep the callback fast, deterministic and free of shared package state, or the engine's coverage feedback and its saved failures stop meaning anything.

code

go · 13 lines
go
f.Fuzz(func(t *testing.T, query string, limit int) {
	parsed, err := ParseQuery(query, limit)
	if err != nil {
		return // an error is an acceptable outcome for hostile input
	}
	again, err := ParseQuery(parsed.String(), limit)
	if err != nil {
		t.Fatalf("printed form no longer parses: %q", parsed.String())
	}
	if again.String() != parsed.String() {
		t.Fatalf("round trip changed %q", parsed.String())
	}
})

go deeper

for a junior

Know the baseline: the engine fails the target on any panic, so 'this input must not crash my code' is a property you get without writing a single assertion.

for a middle

Explain why an expected-value assertion cannot work on generated input, and name the standard property shapes — no panic, round trip, and declared bounds.

for a senior

Show you have done this on real untrusted-input code: early return on legitimate parse errors, bounded allocations, a deterministic self-contained callback, and a judgment about which invariant is worth encoding first.

for a principal

Be ready to argue where property assertions pay for themselves against example-based tests, and what standard the team holds any parser on the trust boundary to before it ships.

## The core difficulty An ordinary test knows its input and therefore knows the answer. A fuzz callback is handed a byte sequence nobody has ever seen, and there is no oracle that says what the parser *should* return for it. So the assertions inside `f.Fuzz` have to be **properties**: statements that are true for every input, valid or not. Four families cover nearly everything worth writing for a parser on a trust boundary. ## 1. It must not crash — and you get this one free The fuzzing engine fails the target on any panic in the callback: an index out of range from a length taken off the wire, a nil dereference on an optional node the parser assumed was present, a slice bounds error from arithmetic that underflowed. You do not assert it; you simply have to not defeat it. The two ways people defeat it are wrapping the call in a `defer`/`recover` to "keep the run going", and catching the panic in the code under test and turning it into an error. Both hide precisely the finding you were fuzzing for. If a parser genuinely wants to recover internally, fuzz the layer below the recovery. The same goes for resource exhaustion. If a parsed length field drives an allocation and the run dies out of memory, that is not a fuzzer artefact — a length taken from untrusted input driving an unbounded allocation is the classic amplification bug. Fix the parser to bound the allocation against the bytes actually available. ## 2. An error is a result, not a failure Most generated input is not a valid query. The parser rejecting it is the system working. The idiom is therefore: ```go parsed, err := ParseQuery(query, limit) if err != nil { return } ``` A callback that instead calls `t.Fatal(err)` reports a "crasher" within milliseconds, on an input that proves nothing, and the search never gets past it. Reserve failure for a panic, a violated invariant, or an error on an input that should have been accepted — for example, one your own printer produced. The mirror-image mistake is asserting on the error's *text*. Error strings are not part of the contract, and pinning them turns every message improvement into a fuzz failure. ## 3. Round trip If the type has a textual form, the strongest cheap property is that parsing and printing are inverse on the accepted subset: parse an input, print it, parse the printed form again, and require the two results to be equal. This catches a large family of real bugs — an escape that is consumed but not re-emitted, a quoted identifier whose quoting is lost, a Unicode form that normalises on one side and not the other, precision quietly dropped from a numeric literal. Note the direction. `parse(print(parse(x))) == parse(x)` is the honest statement; `print(parse(x)) == x` is not, because the parser legitimately normalises whitespace and casing. ## 4. Declared invariants and bounds These are the properties specific to your domain, and the ones a security engineer cares about most: - a parsed `LIMIT` never exceeds the caller's cap, whatever the query text claimed; - the AST's depth is bounded, so deeply nested parentheses cannot become unbounded recursion; - every string in the result is a slice of, or derived from, the input — no field left pointing at a scratch buffer that gets reused; - no field escapes the trust boundary un-escaped, if the parsed form is later rendered into another language. A fifth family, worth the effort when you have it, is **differential**: run a second, obviously-correct-but-slow implementation on the same input and compare. That is how you fuzz a rewrite against the implementation it replaces. ## The callback's own contract Three operational rules make the difference between a target that finds things and one that burns CPU. **Fast.** The callback runs millions of times. Anything that touches the network or filesystem, or that allocates heavily by design, collapses the iteration rate and therefore the depth of the search. **Deterministic.** The same input must produce the same outcome. Reading the clock, using unseeded `math/rand`, letting map iteration order leak into output, or carrying state between invocations in a package-level variable all break this — and when it breaks, a failure the engine saved may not reproduce, and coverage gets attributed to the wrong input. **Self-contained.** Because inputs run in worker processes and in an order you do not control, anything shared between calls is both a correctness hazard and a source of false findings. ## What good looks like in review When a colleague sends you a fuzz target for a parser that reads untrusted input, the questions worth asking are: does an error return end the call quietly; is there at least one invariant beyond "does not panic"; is there any `recover` in the path; does anything in the callback touch time, randomness, the disk or package state; and is there a bound on how much memory a single input can make the parser allocate.

  • Why is `if err != nil { t.Fatal(err) }` inside a fuzz callback usually wrong?
    Almost every mutated input is invalid, so a returned error is the parser doing its job. Failing on it means the engine reports a crasher within milliseconds on an input that proves nothing, and the search never explores further. Return early instead, and reserve failure for a panic, a violated invariant, or an error on input that should have been accepted.
  • The callback allocates a slice sized from a parsed length field and the run dies out of memory. Is that a real bug?
    Almost always yes. A length taken from untrusted input driving an unbounded allocation is the classic amplification bug, and the engine just found it. The fix belongs in the parser: bound the allocation against the bytes actually remaining, or reject the length outright. Muting the target, or capping the value only inside the callback, hides a defect that a real client can trigger.
  • What makes a fuzz callback non-deterministic, and why does the engine care?
    Reading the clock, unseeded `math/rand`, map iteration order leaking into output, network or filesystem access, or state carried between invocations in a package variable. The engine attributes coverage and failures to specific inputs; if the same input behaves differently each time, a saved failure may not reproduce and the search spends its budget chasing noise.
  • When is a differential assertion worth writing instead of an invariant?
    When a second implementation already exists and is trusted — the parser you are replacing, or a deliberately simple reference. Comparing the two on every generated input is the strongest oracle you can get, and it is the standard way to fuzz a rewrite. It costs the runtime of the slow implementation on every iteration, so it suits a scheduled run better than a fast loop.

saying these in an interview costs you the question

  • Asserts an exact expected output for generated input
  • Calls t.Fatal whenever the parser returns an error
  • Thinks a panic has to be asserted explicitly
  • Recovers panics inside the callback to keep the run going
  • Uses the clock or unseeded randomness inside the callback
  • Pins assertions to the text of an error message