skip to content

Two Go services share a gob cache file; which struct changes on one side break the other's decode?

level: seniorimportance: nice to knowfreq 24%

answer

  1. matched by name, not by position
  2. adding a field is the easy case
  3. a rename is a delete plus an add
  4. some failures never return an error
  5. keep yesterday's file and decode it in CI

basics

~20 s

Adding or removing an exported field is tolerated: gob matches fields by name, discards unknown ones and leaves missing ones untouched. Renames silently lose data, incompatible type changes fail at decode, and relocating a registered type breaks interface-typed fields.

solid answer

~50 s

Sort the changes into three buckets. **Safe**: adding a field, or removing one. gob matches by field name, so a field on the wire with no counterpart is discarded and a field in the destination with no counterpart is left as it was. **Loud**: an incompatible type change — signed to unsigned, integer to float, a value too large for the receiving width — fails the decode with an error. So does an interface-typed field whose concrete type was renamed or moved, since the registered name no longer resolves. **Silent, and the dangerous bucket**: renaming a field is a delete plus an add, so the old data lands nowhere and the new field stays at its zero value with no error at all. The defence is a canary: keep a gob file written by the deployed version in the repo, decode it into today's structs in CI, and assert field values rather than just `err == nil`.

go deeper

for a junior

Know the basic rule: gob matches struct fields by name rather than by position, so adding a field does not invalidate data written before it existed.

for a middle

Sort the changes into safe, loud and silent — additive changes are tolerated, incompatible type changes error, and renames or removals lose data without any error at all.

for a senior

Own the mitigation as well as the mechanism: pin registered names, keep a golden file from the deployed version in the repository, and decode it in CI with assertions on the values rather than on the error.

for a principal

Treat the exported field names of a persisted gob struct as an interface between two deploy schedules. Decide who may change them, and how a shared cache is versioned or discarded when someone must.

## The setting One service writes a gob cache file; another reads it. They deploy on different schedules, and the file outlives both binaries. Whoever inherits either service needs to know which edits to the shared struct are free, which will page someone, and which will quietly corrupt results. ## What gob matches on gob matches struct fields **by name**, not by position or by declaration order. The transmitted type descriptor names each exported field, and the decoder lines those names up against the fields of the type it was asked to decode into. Fields on the wire that the destination does not declare are discarded. Fields the destination declares that the record does not carry are left exactly as they were. Neither is an error. That one rule generates most of the behaviour below. ## Safe changes **Adding an exported field.** Old readers ignore it. New readers reading old data leave it at the zero value. Both directions work, which makes additive evolution the natural way to change a gob struct. **Removing a field.** Old data still carrying it is discarded on read. New data simply omits it, and an old reader leaves its copy alone. **Reordering fields.** Irrelevant — matching is by name. **Renaming the Go struct type itself** (not its fields). For plain struct values the type name is not what the match turns on. The exception is a type registered for an interface-typed field, where the registered name *is* the identifier; that case is under "loud" below. **Switching a field between `T` and `*T`.** gob does not transmit pointers; it flattens them and sends what they point to, so the wire representation is unchanged by the indirection. ## Loud changes — they fail with an error **Incompatible numeric changes.** gob will move an integer into a differently sized integer variable, and a float into a differently sized float variable, but the categories have to match and the value has to fit. Signed to unsigned, integer to floating point, or an arriving value too large for the receiving width all fail the decode. **Changing a field's kind.** A field that was a string and is now a struct, or a slice that became a map, is a mismatch the decoder reports rather than papers over. **Moving or renaming a type registered for an interface-typed field.** The registered name is what the decoder looks up to allocate the concrete value, and by default that name is derived from the type's import path and name. Change either and old records name a type nothing maps to. Pinning the name explicitly at registration time is what prevents this. Loud failures are the good kind: they surface on the first read, and they surface as an error you can alert on. ## Silent changes — the ones that hurt **Renaming a field.** From gob's point of view this is one field deleted and a different field added. Old records carry `Hits`; the new struct declares `HitCount`. The old value is discarded, the new field is never written, and the decode returns `nil`. Downstream you see a counter that is suddenly always zero, with nothing in the logs. **Removing a field the reader still relies on.** Same shape: the destination field keeps whatever it had — its zero value on a fresh variable — and nothing is reported. The general lesson: `err == nil` from a gob decode is not evidence that the record was understood. It is evidence that nothing structurally impossible was attempted. ## The defence: a canary decode in CI Because the dangerous bucket is silent, the check has to assert values, not errors. The cheap, effective version: 1. Check a small gob file into the repository — a handful of records produced by the version currently deployed, with every field set to a distinctive non-zero value. 2. Add a test that decodes it into today's structs and asserts each field equals the value it was written with. 3. Regenerate the file deliberately, as an explicit commit, whenever the format changes on purpose. A rename or a dropped field fails that test immediately, on the pull request that introduced it, instead of in whatever service reads the cache next week. An incompatible type change fails it too, just more noisily. It costs a few dozen lines and it is the only thing that catches the silent bucket. The complementary check — whether the *old* binary can still read *new* data — needs the old binary, so in practice it is handled by policy rather than by a test: keep changes additive, pin registered names, and if the format genuinely has to change incompatibly, version the cache path so old and new files never meet. ## What to tell the next maintainer The exported field names of that struct, plus any pinned registration names, are the format. They are an interface between two deploy schedules, and they should be edited with the same care as a public API — which, for a file both services read, is exactly what they are.

  • Which of these failures are loud and which are silent?
    Loud: an incompatible type change, an arriving value too wide for the destination, and an interface-typed field whose registered concrete-type name no longer resolves. Silent: a renamed field and a removed field, both of which leave the destination at its zero value and return no error. That split is why the CI check has to assert field values rather than just check the error.
  • How do you build the canary check concretely?
    Commit a small gob file written by the currently deployed version, with every field set to a distinctive non-zero value, and add a test that decodes it into today's structs and asserts each field. Regenerate the file only as a deliberate commit when the format changes on purpose, so an accidental rename shows up as a failing test on the pull request.
  • Is changing a field from int to *int a wire-compatible change?
    Yes. gob does not put pointers on the wire — it flattens them and transmits the value they point to — so the indirection is invisible to the format. What does change is the meaning of absence on your side: a nil pointer is the field's zero value and is therefore omitted, which is often exactly why the change was made.
  • What if the two services genuinely need incompatible formats?
    Version the storage rather than the struct: write to a new cache path or a new file name, let the old readers keep reading the old path, and retire it once nothing reads it. A shared file with two incompatible interpretations has no safe transition; two files with one interpretation each does.

saying these in an interview costs you the question

  • Assumes gob errors on any struct change
  • Renames a field and expects old data to follow
  • Trusts err == nil as proof the decode was correct
  • Thinks field order on the wire decides matching
  • Forgets a stored file outlives the binary that wrote it