Nullability and Optionality

## Three states, not two A field can reach a reader in more states than most code models. Take a profile record carried as a JSON object: - **Absent** — the key is not in the object at all. The sender made no statement about this field. - **Explicitly null** — the key is present and its value is the null literal. The sender states, on the record, that the field has no value. - **Empty** — the key is present with a zero-length value: `""`, `[]`, `{}`. The field has a value, and that value is empty. - **Default-valued** — the key is present with the type's zero: `0`, `false`. Another ordinary value, and a very common one. The first is a statement about the **message**; the others are statements about the **value**. Everything else follows from that split. Absence is silence, null is an assertion of no-value, and empty and default are values that merely look like nothing. ## Why the difference is worth money The distinction pays off wherever a message is allowed to be partial. If a client sends only the fields it is changing, then absence has to mean *leave this alone* — otherwise the client could never change one field without resending the whole record. But once absence is spoken for, the request needs some other way to say *clear this field*, and an explicit null is the obvious candidate. Collapse the two and the endpoint becomes unable to express one of its two most ordinary operations. It also matters for validation. A rule like "nickname must be at most 30 characters" applies to a present value; it says nothing about an absent field. A rule like "a nickname, once set, may not be blanked" is about the null case only. Writing both against a single nullable variable produces a validator that cannot tell a client who is changing nothing from a client who is clearing something. ## Which encodings keep which distinction | Encoding class | Absent vs present | Explicit null | Empty vs unset | |---|---|---|---| | Self-describing text (JSON-style objects) | Kept — the key is there or it is not | Kept — a first-class literal | Kept — `""` is a present value | | Schema-driven record encodings with a fixed field list | Not expressible within one schema version; absence exists only across versions | Only if the field's type is a union that admits null | Kept, because every field is written | | Tag-based binary encodings that skip default values | Kept via the tag's presence, unless the value equals the type's default | Usually not a value; modelled by a wrapper or a presence bit | Collapsed for default-like values unless presence is tracked separately | The row that surprises people is the last one. If an encoder omits any field whose value equals the type's default, then a field set to the empty string is simply not written, and the reader reconstructs the empty string from the schema. Unset and empty arrive identical, because on the wire they *are* identical. ## What the decode step does to it Even when the bytes carry all three states, the layer that turns bytes into an in-memory object usually flattens them: 1. A missing key becomes the same nullable value as an explicit null. 2. That nullable is handed to business code, which now sees two states where three arrived. 3. The information is not recoverable later — by the time the object exists, the bytes are gone. The fix is to keep presence and content as two separate facts: a presence flag alongside the value, a wrapper object whose own presence is the signal, or a list of the field paths the sender intends to update. Each keeps the reader able to answer *did they mention it?* separately from *what did they say?* A second trap is the intermediary. A service that decodes a body, maps it to objects and re-encodes it will emit whatever its encoder emits — and an encoder that drops nulls turns an explicit null into an absence somewhere in the middle of the path. The distinction is only as strong as the weakest hop. ## What to watch for - Do not treat empty as a synonym for absent. An empty string is a value someone chose to send. - Do not assume every encoding can say null; in some, null is a type you must opt into. - Decide, once, what each state means for a given endpoint, and write it into the contract rather than leaving it to each reader. - If only two of the three states are meaningful for a field, say so in the contract and reject the third instead of silently reinterpreting it.