skip to content

An output column holds 90 absent cells but no input column had any — what could have produced them?

level: middleimportance: should knowfreq 42%

answer

  1. absence can be manufactured, not inherited
  2. zero over zero leaves a hole
  3. labels that met no partner
  4. one marker hides three origins
  5. count the holes per step

basics

~20 s

Two sources manufacture holes out of present data: an operation with no defined answer, such as a zero divided by a zero, and an alignment that pairs row labels present in only one operand. Under a single-marker design both look identical to an unrecorded value.

solid answer

~50 s

Absence in an output does not have to have come from absence in an input. Two mechanisms manufacture it. The first is arithmetic with no defined answer — a zero divided by a zero is the usual culprit, and an even root or a logarithm of a value outside the function's domain does the same. The second appears only where the operands carry their own row labels: if the operator lines the two up on the union of those labels, a label present in one operand and not the other yields a result cell with nothing behind it, so holes appear from two complete inputs. Where the design uses a single marker for all of this, the absence test answers true for a genuine data gap, a division bug and an alignment miss alike, and nothing in the column tells them apart.

go deeper

for a junior

Know that an output can contain empty cells even when every input was complete, so finding one does not prove the source data was incomplete.

for a middle

Name the two manufacturers: an operation with no defined answer on present inputs, and an alignment on row labels that leaves one side without a partner at some label.

for a senior

Show the instrumentation — a per-column absent-cell count recorded at every step — and explain why a single check at the end cannot attribute a hole to the step that created it.

for a principal

The judgment is how much provenance to pay for: recording which step emptied a cell costs discipline on every transform and buys the ability to tell a computation bug from a supplier gap months later.

## Absence is not always inherited The reflex on seeing empty cells in a derived column is to blame the source data. That reflex is wrong often enough to be worth an interview question, because two mechanisms make absence out of inputs that were complete, and the repair for each is entirely different from the repair for a genuine gap. ## Manufacturer one: the arithmetic had no answer Some operations are simply undefined on some present inputs. The canonical case is a **zero divided by a zero**, which has no numeric answer at all — not infinity, not zero, no answer. Others behave the same way: - an even root of a negative value, where no real result exists; - a logarithm of zero or of a negative value, outside the function's domain; - some inverse trigonometric and statistical functions asked for arguments outside their valid range. What the operation returns is a value that means "this computation has no answer", and here is the trap: **under a design whose only absence marker is the reserved pattern the floating-point format sets aside for exactly this situation, that result and an unrecorded observation are literally the same bits.** The absence test answers true for both. Nothing in the column distinguishes a division bug from a gap in the source. Designs that carry a distinct never-recorded marker *separate* from the undefined-arithmetic value do let two different predicates tell them apart, and only one of the two means "go ask the producer". That is one of the real, load-bearing differences between the tools in this space, and it is worth naming rather than assuming. ## Manufacturer two: operands that did not meet The second mechanism is invisible if you have only ever worked with plain positional rectangles of numbers. Where both operands carry row labels of their own, an element-wise operator does not pair cell 1 with cell 1. It first reconciles the two label sets, typically by taking their union, and then computes. A label that appears in one operand and not the other therefore has a value on one side and nothing on the other — and propagation does the rest. The result carries a hole at a label for which neither input had a hole. | | Label-carrying operands | Plain positional rectangles | |---|---|---| | How positions are paired | By reconciling the two label sets | Strictly by position in the rectangle | | Result when a label appears once | An absent cell at that label | Cannot arise; there are no labels | | Result when shapes do not correspond | The union is longer than either input | The operation is rejected outright | | Can absence be manufactured? | Yes | No | This is the one route by which absence appears in a dataset that genuinely had none, and it accounts for a surprising share of "where did these holes come from" tickets. ## Telling the three origins apart There are three ways a cell in an output can be empty, and diagnosing the right one changes what you do next: 1. **Inherited** — an operand was absent and the operator propagated it. Fix upstream, or decide how incomplete records are handled. 2. **Undefined arithmetic** — both operands were present and the operation had no answer. Fix the expression: guard the divisor, constrain the domain, or decide what the undefined case should mean. 3. **Unmatched label** — both operands were complete but did not cover the same labels. Fix the alignment, or make the mismatch an explicit failure rather than a quiet hole. Under a single-marker design, the column itself cannot testify to which of the three happened. Only the computation's own history can. ## Instrumentation, not archaeology - **Record a per-column count of absent cells after every step.** The step where the count jumps owns the explanation, and the record costs one number per column. - **Guard the divisor rather than inspecting the result.** A check on the denominator before the division distinguishes cases 1 and 2 for free; a check on the output cannot. - **Compare the label coverage of two operands before combining them**, so an unmatched label is a decision rather than a discovery. - **Do not reach for a cleaning step first.** Standing the holes up with a chosen value before you know which manufacturer produced them destroys the only evidence you had. - **Say which design you are on.** "An empty cell means nobody measured it" is true only where the never-recorded marker is distinct from the undefined-arithmetic one; elsewhere it is an assumption, and it is the assumption that makes a division bug look like a supplier problem.

  • Under a design with one marker for everything, what evidence survives to tell a data gap from a division bug?
    Nothing in the column. The marker is identical and the absence test answers true for both, so the data cannot testify. Only the computation's history can — the per-step absent-cell counts, or the step re-run in isolation against its own inputs. That is the argument for recording those counts while the pipeline runs rather than trying to reconstruct them afterwards.
  • Does an operation on two plain positional rectangles ever manufacture absence by alignment?
    No. Positional operands are paired cell to cell by their place in the rectangle, so there is no unmatched label that could produce an empty result; if the shapes do not correspond, the operation is rejected outright instead. Alignment-manufactured absence is a property of operands that carry row labels of their own, and it is one of the sharper differences between the two kinds of object.

saying these in an interview costs you the question

  • Assumes every hole in an output came from a hole in an input
  • Says an undefined arithmetic result raises rather than producing a value
  • Treats a division bug and a gap in the source as the same finding
  • Expects positional rectangles to manufacture holes by alignment too
  • Checks the absent-cell count only at the end of the pipeline