In a Go patch API, how do you tell an omitted JSON field from an explicit null when decoding?
answer
- absent and null both leave nil
- a double pointer does not rescue it
- the method runs only for present keys
- and it runs for null too
- presence is a flag the type sets
basics
~20 sA *T field cannot do it, because encoding/json leaves it nil for an absent key and also sets it to nil for an explicit null. Three states need a value-typed wrapper with an UnmarshalJSON method, a json.RawMessage field, or a first pass into map[string]json.RawMessage.
solid answer
~50 sThe natural reach is `*T`, and it gives only two states: absent leaves the field nil because the decoder never visits it, and `null` sets the pointer to nil as well. `**T` does not help either, for the same reason — `null` nils the outer pointer. What does work is the fact that `encoding/json` calls a type's `UnmarshalJSON` **only for keys that appear** and **does call it for null**. So a value-typed wrapper — `struct { Set, Null bool; Value T }` with `UnmarshalJSON` on the pointer receiver — records presence by being invoked at all and records null by inspecting the bytes. `json.RawMessage` gives the same three states for free: nil means absent, the literal bytes `null` mean explicit null, anything else is a value to decode in a second pass. Alternatively decode once into `map[string]json.RawMessage` to learn which keys were present. Then the design decision has to be written down: absent means leave unchanged, null means clear.
code
go · 14 linestype Optional[T any] struct {
Set bool // the key appeared in the JSON object
Null bool // the key appeared and its value was null
Value T
}
func (o *Optional[T]) UnmarshalJSON(b []byte) error {
o.Set = true
if string(b) == "null" {
o.Null = true
return nil
}
return json.Unmarshal(b, &o.Value)
}go deeper
Know that a pointer field is the usual way to tell an unset field from one that was sent, and that a missing key simply leaves the Go field at its zero value.
Explain why a pointer tops out at two states, and describe the decoder behaviour a wrapper type relies on: the method runs only for keys that appear, and it runs for null as well.
Walk the whole failure: an omitted field read as zero, a reconcile loop acting on it, and a log line that cannot distinguish a client's intent from a decoding default. Then give the representation and the tests that pin it.
Own the contract, not the struct. Publish what absent, null and a zero value each mean before any handler is written, because every client encodes those assumptions and no compiler will flag a later change of mind.
## The problem A partial-update endpoint — a controller that reconciles a declared spec, a PATCH handler, a config overlay — has to distinguish three things a client can say about a field: 1. **absent** — I am not talking about this field; leave it as it is. 2. **null** — I am talking about this field; clear it. 3. **a value** — set it to this, including when "this" is `0`, `false` or `""`. Get it wrong and the failure is a wrong zero value: a spec that omits `replicas` is read as `replicas: 0`, the reconcile loop obediently scales the workload to nothing, and the only trace is a log line saying it set replicas to 0 — which is true, and useless, because the log cannot say whether the client asked for it. ## Why the obvious answers fail **A plain value type** (`Replicas int`) has one state. An absent key leaves the zero value, and the zero value is a legal input, so absent and `0` are indistinguishable. **A pointer (`*int`)** has two states, which is enough for absent-versus-value and is the right tool when `null` is not part of your contract. It cannot express three: `encoding/json` sets a settable pointer field to `nil` when it decodes `null`, which is precisely the state an absent key leaves it in. **A double pointer (`**int`)** is the reflex fix and it does not work in `encoding/json`. The field itself is a pointer, so `null` sets *it* to nil, giving the same nil the absent case gives. Anyone who proposes `**T` here has not run it. ## The mechanism that does work Two facts about the decoder combine into a solution: - `UnmarshalJSON` is called **only for keys present** in the JSON object. An absent key means the field is never visited at all. - `UnmarshalJSON` **is** called when the value is `null`, receiving the four bytes `null`, provided the field is a value of the type rather than a settable pointer to it. So a value-typed wrapper can record everything: ``` type Optional[T any] struct { Set bool Null bool Value T } func (o *Optional[T]) UnmarshalJSON(b []byte) error { o.Set = true if string(b) == "null" { o.Null = true return nil } return json.Unmarshal(b, &o.Value) } ``` `Set` false means absent. `Set` true with `Null` true means an explicit null. `Set` true with `Null` false means a value, and that value may legitimately be the zero one. Note the field must be declared as `Optional[T]`, not `*Optional[T]`: a settable pointer field would be nil'd by `null` before the method was reached. ## Two lighter alternatives **`json.RawMessage`.** It is a `[]byte` with `UnmarshalJSON` defined, so a `json.RawMessage` field is nil when the key was absent, holds the bytes `null` when the key was explicitly null, and holds the raw value otherwise. Three states, no code of your own, at the cost of a second `json.Unmarshal` when you want the value. **`map[string]json.RawMessage`.** Decode the document once into a map to learn the key set, then decode again into the typed struct. It answers the presence question for every field at once and is convenient when the handler wants to iterate over "which fields did the client mention", but it loses the typed struct's field names as the source of truth and costs a second parse of the whole document. ## Encoding is the other half A wrapper type needs a `MarshalJSON` too if the same struct is used for responses, and it has to decide what an unset field encodes as. Usually the answer is: do not reuse the patch type for responses. A response has two states, not three, and a separate struct with plain fields or pointers says that clearly. ## The part that is not code The representation is the easy half. The contract is the decision someone has to make and publish: does `null` clear a field to its zero value, remove it entirely, or restore a default? Is omitting a required field an error or a no-op? May a client send `null` for a field that has no null representation in storage? Those answers belong in the API documentation before the struct is written, because every consumer encodes an assumption about them and changing the answer later is a breaking change that no compiler will catch. ## What to say in an interview Start by ruling out `*T` with the reason — `null` nils it exactly as absence does — then give the decoder facts about `UnmarshalJSON` being called only for present keys and also for `null`, then the wrapper. Mention `json.RawMessage` as the low-effort version. Finish on the contract question, because that is the part the interviewer is actually probing for at senior level.
- Why does a **T field not solve this in encoding/json?Because the field is itself a pointer, and the decoder's rule for `null` is to set a settable pointer to nil. So `{"x":null}` leaves the `**T` field nil, which is exactly the state an absent key leaves it in. The extra level of indirection is never allocated, so no inner nil is ever distinguishable from the outer one.
- What is the cheapest way to get three states without writing a type?Declare the field as `json.RawMessage`. It is nil when the key was absent, holds the literal bytes `null` when the client sent null, and otherwise holds the raw value bytes, which a second `json.Unmarshal` turns into the typed value. The cost is that extra decode step and a field type that says less about what it holds.
- Beyond the decoding, what has to be decided before this struct is written?What each of the three states means in the domain: whether null clears to the zero value, deletes the field, or restores a default; whether omitting a field is ever an error; and whether every field even has a null meaning. Consumers bake those answers into their clients, so changing one afterwards is a breaking change with no compile-time signal.
- How would you catch a wrong-zero-value bug like this before it reaches production?Table-driven decode tests whose cases are exactly the three inputs — key absent, key null, key with the zero value — asserting the decoded state rather than the final effect. The absent-versus-zero pair is the one that regresses when someone simplifies a field back to a plain type, and it is invisible in a test that only ever sends fully populated documents.
A pointer is a light switch with two positions when the question has three answers: nothing said, said nothing, and said something.
saying these in an interview costs you the question
- Claims *T distinguishes an absent field from an explicit null
- Proposes **T without knowing null nils the outer pointer
- Treats a missing field as equivalent to the zero value in a patch
- Thinks UnmarshalJSON is skipped when the value is null
- Ships three-state decoding without documenting what null means