skip to content

In encoding/gob, why does decoding into a reused struct variable leave stale field values?

level: middleimportance: nice to knowfreq 22%

answer

  1. the encoder is lazy about zeros
  2. Decode does not clear anything
  3. absent and zero look identical
  4. declare the destination inside the loop

basics

~20 s

gob omits struct fields holding their type's zero value, and the decoder writes only the fields the stream contains. Fields absent from a record are left untouched, so a reused destination keeps the previous record's values.

solid answer

~50 s

Two rules combine into the bug. First, the encoder drops any struct field that holds the zero value for its type — a `false`, a `0`, an empty string never reaches the wire. Second, `Decode` writes only the fields that *are* present; it does not clear the destination first, so anything not mentioned in this record keeps whatever was already there. Decode a loop of records into one long-lived variable and a record whose `Count` is zero silently inherits the previous record's `Count`. The fix is to give each record a fresh destination — declare the variable inside the loop, or assign the zero struct before each `Decode`. The same rules also mean gob cannot express the difference between "this field was absent" and "this field was zero"; if that distinction matters, model it in the data.

code

go · 13 lines
go
var e Entry // BUG: reused across every record
for {
	err := dec.Decode(&e)
	if err == io.EOF {
		break
	}
	if err != nil {
		return err
	}
	// A record whose Count was 0 did not transmit Count at all,
	// so e.Count still holds the previous record's value.
	use(e)
}

go deeper

for a junior

Remember the two-line rule: gob does not send zero-valued struct fields, and Decode does not clear the value you hand it. Declare the destination inside the decode loop.

for a middle

Explain the omission as a size optimisation and name what it costs: on the wire, absent and zero become the same thing, so the receiver cannot recover the difference.

for a senior

Recognise the bug shape in review — a destination declared above a decode loop — and know why tests miss it: hand-written fixtures rarely contain the zero-valued fields that trigger the carry-over.

for a principal

Note what this means for data you store rather than transmit: a gob record cannot express 'this field was explicitly cleared', so any reset or tombstone semantics has to be modelled in the data itself.

## The two rules ### The encoder omits zero-valued fields When gob encodes a struct, a field whose value equals the zero value for its type is simply not transmitted. `0`, `false`, `""`, a nil map, a nil slice, a nil pointer in a struct field — none of them appear on the wire. This is a size optimisation, and for records that are mostly-empty structs it is a large one. ### The decoder writes only what arrived `Decode` walks the fields present in the incoming record and writes those into the destination. It does **not** zero the destination first, and it does not touch fields the record does not mention. The Go value you pass in is a starting point that gets selectively overwritten, not a blank slate that gets filled. ## Why that produces stale data Put the two together with the very natural optimisation of hoisting the destination out of a decode loop, and you get carry-over. Suppose records look like `{Key string; Count int}`. Record one has `Count: 5`, so `Count` is transmitted. Record two has `Count: 0`, so `Count` is *not* transmitted. Decoding record two into the same variable leaves `Count` at 5. Nothing errors; the value is simply wrong. The symptom in production is characteristic: values that look plausible but belong to the previous record, appearing only on records where a field happened to be zero — which is exactly the case a test with hand-written non-zero fixtures never exercises. ## The fix Give every record a fresh destination. Declaring the variable inside the loop body is the idiomatic form, and it is free — the compiler is perfectly happy to reuse the storage. If you have a reason to keep one variable (a large struct you are deliberately reusing), assign the zero value explicitly before each `Decode`: `e = Entry{}`. This is the one place where the usual Go advice to hoist an allocation out of a loop is actively harmful, and it is worth calling out in code review of any gob decode loop. ## The wider consequence: absent and zero are the same thing Because the encoder drops zero-valued fields, the receiver has no way to distinguish "the sender explicitly set this to false" from "the sender never set it". Both arrive as nothing at all. For a cache record or an internal message that is usually fine — the zero value *is* the default you want. It stops being fine when zero is meaningful: a quantity that was deliberately cleared, a flag that was deliberately turned off, a counter that was reset. When you need that distinction, encode it in the data rather than hoping the format will carry it. A pointer field turns "absent" into a nil pointer distinct from a pointed-to zero. An explicit companion field, or a small enum with a non-zero "cleared" member, does the same job more legibly. The important part is recognising that the format itself will not preserve the difference. ## What it is not It is worth separating this from two nearby things. It is not an error condition — no error is returned, so a decode loop checking only `err != nil` sees nothing wrong. And it is not about unexported fields, which are a separate rule: those never travel at all, whether or not they are zero.

  • Can the receiver tell whether a false bool arrived in the record or was never sent?
    No. The encoder drops any field equal to its type's zero value, so `false`, `0` and `""` never reach the wire and the decoder simply leaves the destination field alone. If that distinction carries meaning, model it in the data — a pointer field whose nil case means absent, or an explicit companion field — and decode into a freshly zeroed destination so the default is unambiguous.
  • Does Decode zero the destination before writing into it?
    No. It writes only the fields the incoming record contains and leaves everything else in the destination exactly as it was. That is precisely why the standard advice is to decode into a variable declared inside the loop rather than one hoisted above it.
  • Would this bug show up in a unit test?
    Usually not, because hand-written fixtures tend to set every field to something non-zero, so every field is transmitted on every record and nothing carries over. It surfaces on real data, where zero values are common. A test that deliberately includes a record with zero-valued fields after a fully populated one catches it.

The decoder is a painter working from a list of walls to repaint. Any wall not on today's list keeps yesterday's colour — and the encoder leaves a wall off the list whenever the new colour is the default one.

saying these in an interview costs you the question

  • Assumes Decode clears the destination first
  • Thinks a zero-valued field is still sent explicitly
  • Believes gob distinguishes absent from false
  • Reuses one destination struct across a decode loop
  • Treats err == nil as proof the record decoded correctly