Why can a reader that substitutes implicit defaults not tell a writer's explicit zero from an unset field?

## The rule that creates the ambiguity An **implicit default** is a rule in the wire contract that says: if the bytes for a field are not present, the reader behaves as if the field carried a particular fixed value. That value is normally the type's zero — `0` for numbers, the empty string for text, `false` for booleans, an empty list for repeated fields, and the first declared member for an enumerated type. The rule is there for a good reason. It means a reader written against a newer schema version can decode an older message without special-casing every field added since, and it keeps messages small, because a field at its default carries no information and need not be written. The cost is that **the decoded value carries no record of which fields were actually on the wire**. ## Where the presence information is destroyed 1. **The writer decides.** It either sets the field to a value that happens to be the default, or does not set it at all. 2. **The encoder may elide.** Several encoding families omit a field whose value equals the default, so the two writer intentions converge to the same bytes. 3. **The reader substitutes.** On decode, the missing field becomes the default in the in-memory value. By the time application code runs, step 3 has already happened. No amount of validation downstream can distinguish the two cases, because there is nothing left to distinguish. ## The values that collide | Field | Implicit default | Real value that collides | Symptom in production | |---|---|---|---| | Discount percentage | `0` | A deliberate zero discount | Cannot tell "no discount applied" from "discount not calculated yet" | | Retry count | `0` | A message never retried | A missing counter reads as a fresh message | | `isSuspended` | `false` | An account explicitly cleared | An unset flag reads as an explicit clearance | | Currency code | empty string | Never a real value | Safe: the default is inert | | Priority (enumerated) | first member | Whatever the first member happens to mean | A new writer's silence reads as that member | The last two rows carry the lesson. The empty string is a safe default for a currency code because no valid currency code is empty — the default is **inert**. The enumerated field is dangerous precisely because its default is a real, meaningful member that somebody's code will act on. ## Why an intermediary makes it worse A hop that decodes a message, changes one field and re-emits it does not forward the writer's silence — it forwards its own **materialised** value. A field the producer never set arrives at the hop as the default, is copied into the outgoing value, and is then written out as an explicit default. What began as "the producer had nothing to say about this field" ends as "an upstream service asserted this value". Downstream logic that was written to treat an absent field as "unknown, go and look it up" now sees an assertion and skips the lookup. ## Designing the ambiguity away You cannot recover presence after the fact, so the work is in the field's design: - **Make the default inert.** Choose an encoding of the domain in which the type's default is a value that means "do nothing" — an empty string for a code, an empty list for a set of overrides, an explicitly declared `unspecified` member as the first enumerated value. - **Never let a defaulted field carry a decision.** If acting on the default and acting on a genuine zero must differ, the difference belongs in a second field that says which operation was requested, not in the value itself. - **Shift the unit so zero is impossible.** A field recording a percentage in basis points where `0` is legal is ambiguous; one recording an override that is only ever written when an override exists is not. - **Do not reach for a wider numeric type.** A larger range does not separate zero from unset; the collision is about the default rule, not about the representable values. ## What an interviewer is listening for A weak answer treats this as a nullability preference. A strong answer names the moment the information is destroyed — default substitution during decode, before any application code sees the message — and follows it through a pipeline, where an omission on one hop becomes an explicit assertion on the next. The strongest answers also say what they would *not* do: adding a parallel "was this set" boolean to every field is a design that grows without bound, so it is better to pick field semantics whose default means nothing at all.