A cell in a typed column always occupies its slot, so how can a tool record that nothing was measured there?
answer
- the slot is always there
- absence has to be encoded somehow
- inside the value, or beside it
- a reference, or the tool's own marker
- the encoding decides which columns qualify
basics
~20 sA typed column stores one fixed form in every cell, so absence must be encoded somewhere: a reserved bit pattern inside the value, a validity bit beside it, the language's empty reference, or a marker the tool defines.
solid answer
~50 sA column is not a bag of cells that can be left out — it is one physical form repeated, with a slot per row, so "nothing here" has to be represented by something. Four designs are in use. A tool can borrow a bit pattern the value format already sets aside — the floating-point format's not-a-number pattern is the usual donor — and read it as "no value". It can keep a validity track beside the values: one bit per row saying whether the value stored next to it counts at all. It can store the host language's own empty reference, which only works where the cells are references to arbitrary things. Or it can define its own typed absence marker, a single value usable in a column of any representation. Which one a tool picked decides which columns can hold a hole, and at what width.
go deeper
Recall that absence is a stored state, not an empty slot: the cell exists, so something has to stand for "no value". Being able to name two of the four encodings is enough at this level.
Explain where each encoding physically sits — inside the value, in a track beside it, as a reference in the cell, or as a marker the library owns — and which kinds of column each one can actually serve.
Show that you reason from the encoding to the observable: what the column's width does, whether a single check finds every hole, and what survives when the data is handed to a tool that uses a different scheme.
Frame it as a commitment: a pipeline that crosses tools inherits whichever absence scheme each one uses, so the team has to decide where absence is normalised and who owns the conversion at each boundary.
## Why "nothing" needs a representation of its own A column in a two-dimensional labelled table — a rectangle of columns where every column has one type and the rows carry an identity of their own — is not a list of independent cells that may simply be omitted. It is **one physical form repeated**: one width, one interpretation, one slot per row. That is what makes whole-column work possible at all, and it is also what makes absence awkward. - The slot exists whether or not a measurement ever arrived. - Reading the column means walking every slot in order, so every slot must decode to something. - Therefore "no value here" is not the absence of data in the storage sense. It is either **a pattern the slot holds** or **a fact recorded somewhere beside the slot**. Every design in this family picks one of those two answers, and there are four encodings in common use. ## The four encodings 1. **A reserved pattern inside the value.** The value format itself sets aside bit patterns that are not ordinary numbers; the floating-point format reserves a not-a-number pattern for undefined results, and a tool can borrow it to mean "no value here". Nothing extra is stored — the hole lives in the number's own bits. 2. **A validity bit beside the value.** A separate one-bit-per-row track says whether the value stored next to it counts at all. The value slot still holds something (usually whatever was there, often a zero), but the track overrides it. 3. **The host language's empty reference.** The language's own "nothing here" object is written straight into the cell. This only works in a column of arbitrary references — a column whose cells are pointers to anything at all. 4. **The tool's own typed absence marker.** The library defines one absence value of its own and allows it in a column of any representation, without borrowing a pattern from the value format and without changing the column's width. ## What each one can serve | encoding | where absence lives | which columns it can serve | the catch | |---|---|---|---| | reserved pattern inside the value | in the value's own bits | only formats that have a spare pattern — in practice, floating-point | a whole-number column has no spare pattern, so it must be re-stored in a form that does | | validity bit beside the value | in a separate track, one bit per row | any column, whatever its width | the presence information is a second thing to carry, read and keep in step | | host language's empty reference | in the cell, as a reference | only a column of arbitrary references | the column is no longer one uniform form, and can collect several different absence-ish objects | | the tool's own typed marker | outside the value, defined by the library | any column, width unchanged | it is the tool's object, so code written against the format's reserved pattern will not recognise it | ## Why this is visible above the storage layer It is tempting to file this as an implementation detail. It is not, because four things a candidate actually observes follow from it: - **whether a whole-number column keeps its integer form** when one value goes unrecorded; - **whether one check finds every hole**, or whether a column has accumulated more than one kind of absence-ish object; - **whether the absence survives a hand-off** to another tool, which may not know the marker it is looking at; - **whether absence can be recorded at all** in a boolean or a time column, where the value format may have nothing spare to lend. Note also that the detection predicate is a purpose-built **absence test** — a predicate that asks, cell by cell, whether the value is absent. It exists precisely because the encodings differ and because comparing a cell against a marker is not the same question as asking whether the cell is empty. ## Finding out which one you are on 1. Put a hole into a narrow whole-number column and look at what the column's declared representation becomes. If it changed, the tool's marker is a pattern borrowed from inside the value. 2. Ask whether the tool exposes a whole-number column that admits absence — a whole-number column that keeps its integer width because absence is tracked beside the values. Its existence means a validity track or a typed marker is available. 3. Ask whether a boolean or a time column can hold a hole. Where it can, absence is not being encoded inside the value. The point to carry into an interview is the first one: absence is not a hole in the data, it is a **representation choice somebody made**, and the rest of the topic — what widths you pay, what a check finds, what survives a hand-off — falls out of which choice is in force.
- Why can a column of arbitrary references end up holding several different absence-ish things at once?Because every cell is a reference, anything can be written into it. One loader can leave the host language's empty reference, a numeric step can leave a reserved floating-point pattern, and the tool's own typed marker can arrive from a third place. Nothing in the column constrains them to one form, so a check written for a single marker finds only its own and reports the column cleaner than it is.
- What does a tool gain by defining its own typed absence marker instead of borrowing one from the value format?One marker serves a column of any representation, so a whole-number, boolean or time column can hold a hole without giving up its own form or width. The cost is that the marker is the tool's object rather than the format's, so anything reading the values through the raw value format — another library, a hand-written loop, an exporter — does not recognise it unless it was written to.
A paper form with pre-printed boxes: you cannot make a box vanish. You either write a special mark inside the box — which only works if the box's own notation has a mark spare — or you tick a separate "not collected" checkbox in the margin, one per box. Inside the value, or beside it.
saying these in an interview costs you the question
- Thinks an absent cell simply stores nothing at all
- Assumes every tool marks absence the same way
- Believes absence is always a bit pattern inside the number
- Thinks a validity bit lives inside the value it describes
- Says any column kind can hold the same marker at no cost