skip to content

One Column, Two Kinds of Thing

One unparseable token in a million rows decides the representation of the whole column, and what happens next - a loud failure or a plausible wrong answer - is not the same everywhere.

on this pageshow

questions

4

A column of a million quantities is built and one entry is a word. Why does that one entry decide how all million are stored?

level: juniorimportance: must knowfreq 70%

answer

  1. storage is chosen per column
  2. no single row can be special
  3. three dispositions, not one rule
  4. widen to anything, refuse, or tag

basics

~20 s

A column carries one representation for all its rows, so no row can be special. The misfit forces a whole-column choice: fall back to a form that holds anything, refuse the value, or tag every row with its kind.

solid answer

~50 s

A column's representation — the one physical form every value in it is stored in — belongs to the column, not to the value. A fixed-width numeric column is a packed buffer: every row occupies the same declared number of bytes, and a word has no encoding there at all. So one non-conforming entry is a decision about all of them, and designs in this space take one of three. Some **fall back** to a holder-for-anything representation, which stores one reference per row and lets each row be a different kind of thing. Some **refuse** the value, because the column's declared type is validated as values are stored, and the program stops at the line that built it. Some carry a **declared union**: the column may hold a value from a small named set of kinds, and each row is tagged with which. Only the first two are common, and only the last two leave the column honest about what is in it.

go deeper

for a junior

Recall that storage is chosen once per column: every row is stored the same way, so a single value that does not fit is a decision about all of them.

for a middle

Explain the packed buffer — the same declared number of bytes per row — and why an entry with no encoding there cannot be an exception. Then name all three outcomes instead of assuming the column widens.

for a senior

Show that you know which outcome a given system produces and how you would find out, because that decides whether a bad value stops at your line or reaches a published number weeks later.

for a principal

The angle is where the decision is made and who pays for it. A column that silently absorbs anything spreads the cost across every consumer forever; refusing concentrates it on one producer once.

## What "representation" means here By **type** in this question I mean the column's run-time representation — the one physical form every value in that column is stored in — not a static annotation a checker reads, and not the type recorded in whatever file the data came from. That representation is a property of the **column**. It is fixed once, and every row is stored under it. The cheapest representations are **fixed-width numeric columns**: every row occupies the same declared number of bytes, laid end to end in one buffer. Reading row *i* is arithmetic — start of buffer plus *i* times the width — and an operation over the column is one machine instruction repeated. Nothing in that layout has anywhere to record "this row is different". A word is not a quantity of any width; there is no encoding of it in a numeric slot. So the question "what do we do with this one value" has no per-row answer. It is a question about the column. ## The three dispositions | disposition | what the column becomes | what happens to the misfit | what it costs | |---|---|---|---| | **Fall back** | a holder-for-anything representation: one reference per row, each row free to be a different kind of thing | kept, untagged, indistinguishable until something inspects it | the uniform buffer is gone, so every later pass inspects each row before it can act on it | | **Refuse** | nothing — the column is never built | rejected at the line that tried to store it | the run stops, loudly, at a known place; somebody must decide what the value was | | **Declared union** | a column allowed to hold a value from a small named set of kinds, each row tagged with which | kept **and** labelled | every consumer must branch on the tag; not every design offers this | The first is the one people meet without choosing it. The second is the one that saves a report. The third is the one candidates most often do not know exists. ## Which one you get is a property of the design, not a law This is the part that separates a candidate who has been burned from one who has read a tutorial: - Designs that **re-infer** a column's representation while it is being built, or when two inputs are combined, resolve to the most general form that holds everything they were given. When no numeric form is general enough, "most general" means the holder-for-anything representation. - Designs that treat a **declared column type** as a contract validated at construction refuse the value instead, and the failure lands on your line rather than in next week's numbers. - Designs that treat a declared type as a starting hint — honoured when the table is built, then re-decided by the next step that returns a new column — give you neither guarantee: nothing refused anything, and the column quietly became something else. So "a column that meets a value it cannot hold widens to hold it" is true of some designs and false of others, and the useful interview move is to say which one you are in and how you would tell. ## Falling back is not widening Two events are easy to confuse and cost very different amounts: 1. **Widening inside the numbers.** The column stays a packed buffer of uniform width; the width just got bigger. The pass over it is still one operation per row. You pay bytes. 2. **Dropping to a holder-for-anything form.** There is no uniform buffer any more. Each row is a reference to a separately allocated value, and every operation has to discover each row's kind before it can do anything. You pay bytes *and* every subsequent pass. Calling the second a "widening" is the classic muddle, and it is why the symptom shows up a week later as a performance mystery rather than as an error. ## What the interviewer is listening for A weak answer is "the tool handles it". A good answer has three parts: - storage is chosen **per column**, so one value that does not fit is a whole-column event; - there are **three** possible outcomes, and which one occurs depends on whether the declared type is enforced, re-inferred, or merely a hint; - the outcome that keeps the program running is also the outcome that stops telling you anything, because a column that holds anything holds the bad value silently alongside the good ones. That last point is the whole reason this material is asked of juniors: the failure mode is not a crash, it is a column that still prints fine.

  • Is falling back to a form that holds anything the same event as a narrow numeric column becoming a wider one?
    No. Widening inside the numbers keeps a packed buffer of uniform width and one machine operation per row; the column is bigger but still typed. Falling back abandons the uniform buffer, so every operation has to ask what each row is first. The first costs bytes; the second costs bytes and every pass afterwards.
  • If a design can refuse the value, how do columns holding two kinds of thing exist at all?
    Refusal is a property of the design, not a law. Where a declared column type is validated as values are stored, the non-conforming value stops the program at that line. Where the declared type is a starting hint the next operation may re-decide, nothing refuses anything and the column quietly becomes something else. Which regime you are in decides whether you get a loud failure or a plausible wrong answer.
  • Which of the three dispositions keeps the information carried by the misfit value?
    The declared union keeps it and says what it is. The fallback keeps it but says nothing, so every consumer rediscovers its kind. Refusal keeps nothing in the column, though the failure itself records that a row was rejected. The choice is largely about who downstream is willing to branch.

A filing drawer whose slots are all cut to one card size. A document that will not fit leaves three options: replace every slot with an envelope that takes anything, so nothing is uniform and every lookup has to open the envelope first; turn the document away at the drawer; or relabel every slot with what kind of thing is inside, so a reader knows before opening. What you cannot do is make one slot different and tell nobody.

saying these in an interview costs you the question

  • Says only the one bad row is stored differently from the rest
  • Assumes every design silently widens, unaware refusal and tagging exist
  • Treats the fallback form as just another numeric widening
  • Thinks the column is only bigger afterwards, not slower
  • Believes a stated column type always guarantees what the column holds
open as a page

Why is a column that stores one reference per row, each of any kind, slower to compute over and not merely wider?

level: middleimportance: must knowfreq 60%

basics

~20 s

Because the single typed pass is gone. Over a packed buffer the same machine operation repeats for every row; once each row may be anything, each row costs a dereference, a kind check, an operation lookup and a freshly allocated result.

open as a page

What does a column declared to hold one of a small named set of kinds give you that a hold-anything column does not?

level: middleimportance: should knowfreq 42%

basics

~20 s

A closed, named set of kinds and a per-row tag. Consumers can branch over every case that exists, and an unexpected fourth kind is refused rather than absorbed, where a hold-anything column accepts everything and records nothing about what arrived.

open as a page

Two million rows carry a quantity column where a few dozen entries are not quantities; which disposition do you take, and what does each cost downstream?

level: seniorimportance: should knowfreq 46%

basics

~20 s

Decide at the boundary, not downstream. Refusing stops the batch at a known line and keeps the column typed; falling back keeps the batch running and taxes every later pass; a tagged union keeps the evidence and makes every consumer branch.

open as a page