Why is a column that stores one reference per row, each of any kind, slower to compute over and not merely wider?

middlemust knowfreq 60%

answer

the typed pass is what was lost
the decision moved inside the loop
dereference, kind check, operation lookup
what a callback receives decides its speed

basics

~20 s

Because the single typed pass is gone. Over a packed buffer the same machine operation repeats for every row; once each row may be anything, each row costs a dereference, a kind check, an operation lookup and a freshly allocated result.

solid answer

~50 s

A fixed-width column is one contiguous buffer of equal-size values, so an operation over it is one instruction chosen once and repeated — the work per row is the arithmetic itself. A holder-for-anything column stores a reference per row to a separately allocated value, and the kinds are not known in advance. For every row the engine must follow the reference, ask what kind of thing it found, select an operation appropriate to that kind, run it, and allocate somewhere to put the answer. That is per-value dispatch, and it is several times the work of the arithmetic it surrounds. Two second-order effects follow: results usually come back in the same representation, so the cost propagates, and comparison and ordering stop being a single rule — values of different kinds may compare by per-kind rules or refuse to compare at all.

A fixed-width column is one contiguous buffer of equal-size values, so an operation over it is one instruction chosen once and repeated — the work per row is the arithmetic itself. A holder-for-anything column stores a reference per row to a separately allocated value, and the kinds are not known in advance. For every row the engine must follow the reference, ask what kind of thing it found, select an operation appropriate to that kind, run it, and allocate somewhere to put the answer. That is **per-value dispatch**, and it is several times the work of the arithmetic it surrounds. Two second-order effects follow: results usually come back in the same representation, so the cost propagates, and comparison and ordering stop being a single rule — values of different kinds may compare by per-kind rules or refuse to compare at all.

code

pseudocode · 11 lines

pseudocode

# one representation, fixed width: the operation is chosen once
op = machine_add_for(column.representation)
for i in 0 .. n-1:
    out[i] = op(buffer[i], 1)          # adjacent bytes, same instruction every row

# one reference per row, any kind: the operation is chosen n times
for i in 0 .. n-1:
    v  = deref(slot[i])                # jump to wherever this value was allocated
    k  = kind_of(v)                    # what is this row, actually?
    op = lookup_add_for(k)             # may not exist -> fail here, mid-pass
    out[i] = allocate(op(v, 1))        # a fresh allocation, and a reference to it

go deeper

for a junior

Recall the difference between a column being bigger and a column being slower. Here both happen, and the slowdown is the one people miss.

for a middle

Walk the per-row steps out loud: follow the reference, establish the kind, pick an operation, allocate the result. Then contrast that with one instruction chosen once for a packed buffer.

for a senior

Demonstrate that you would look for this when a pipeline that used to finish in seconds now takes minutes with no change in row count, and that you would say which representation the column ended up in before proposing a fix.

for a principal

The tradeoff is where this cost is allowed to exist at all. Permitting the fallback anywhere in a shared pipeline means every consumer inherits it, so the question is whether the boundary should refuse instead.

What a typed pass is

When a column has one representation of fixed width, its values sit end to end in one block of memory. An operation over that column is decided once: the engine knows every entry is the same kind and the same size, so it picks the machine operation before the loop starts and then repeats it. The per-row cost is the arithmetic and nothing else, and because the values are adjacent the memory system can fetch them in bulk and work ahead.

That is the pass you lose. Everything below is what replaces it.

What per-value dispatch actually does

When the column is a holder-for-anything representation — one reference per row, each row free to be a different kind of thing — the engine cannot decide anything before the loop. For each row it must:

follow the reference to wherever that value was separately allocated;
discover what kind of thing it is;
select an operation valid for that kind, or fail;
run the operation;
allocate somewhere for the result and store a reference to it.

Steps 1, 2, 3 and 5 are pure overhead; only step 4 is the work you asked for. And step 1 is not free in a second way: the values were allocated at different times and live in different places, so the traversal jumps around memory rather than streaming through it.

	uniform typed buffer	one reference per row
operation chosen	once, before the loop	once per row
memory access	sequential, predictable	scattered, one indirection per row
result	written into a packed buffer	a new allocation per row
a value of the wrong kind	cannot exist	discovered mid-pass, may fail there

The claim to be careful with

"A function you hand in is called once per value, so it is always slow" is the sentence to avoid, because it attributes the cost to the wrong thing. Name what the callback receives. Handed one value at a time, over a column where each value's kind must be established first, it pays dispatch on every row — that really is the cost this representation buys you. Handed the whole column at once, the same surface name does one pass and runs at full speed. And over a uniform typed column, many operations involve no callback at all: they are a compiled pass the engine chose in advance. So the slowness belongs to per-value dispatch over an untyped column, not to the idea of supplying a function.

The same care applies in the other direction: not every design falls back like this. Where a declared column type is enforced, the column never becomes a holder for anything, and this cost never arises — the run stopped earlier instead.

Second-order costs

The result inherits the representation. An operation over such a column usually hands back another one, so the next step pays again. The cost is not a one-off tax on the step that noticed.
Ordering stops being one rule. A sort over a uniform numeric column is a single comparison repeated. Over a column of mixed kinds, comparisons are per-kind, may follow rules you did not choose, or may refuse outright — and if they do not refuse, the ordering can be quietly meaningless.
Aggregates may still produce a number. Something that combines values kind by kind can return a plausible figure computed over a subset it chose itself, which is far worse than an error.
Measurement misleads. A shallow look at the column sees a reference per row and reports a small, uniform figure; the values it points at are elsewhere. The reported size and the real size are different numbers.

What the interviewer wants to hear

The distinction between bigger and slower is the whole question. A candidate who says "it uses more memory" has noticed the symptom people mention and missed the mechanism. The answer worth hearing is: the column lost the property that let one decision cover every row, so the decision now happens a million times, and it happens in the middle of a pass that also has to chase a reference to find out what it is deciding about.

## What a typed pass is When a column has one representation of fixed width, its values sit end to end in one block of memory. An operation over that column is decided **once**: the engine knows every entry is the same kind and the same size, so it picks the machine operation before the loop starts and then repeats it. The per-row cost is the arithmetic and nothing else, and because the values are adjacent the memory system can fetch them in bulk and work ahead. That is the pass you lose. Everything below is what replaces it. ## What per-value dispatch actually does When the column is a holder-for-anything representation — one reference per row, each row free to be a different kind of thing — the engine cannot decide anything before the loop. For **each row** it must: 1. follow the reference to wherever that value was separately allocated; 2. discover what kind of thing it is; 3. select an operation valid for that kind, or fail; 4. run the operation; 5. allocate somewhere for the result and store a reference to it. Steps 1, 2, 3 and 5 are pure overhead; only step 4 is the work you asked for. And step 1 is not free in a second way: the values were allocated at different times and live in different places, so the traversal jumps around memory rather than streaming through it. | | uniform typed buffer | one reference per row | |---|---|---| | operation chosen | once, before the loop | once per row | | memory access | sequential, predictable | scattered, one indirection per row | | result | written into a packed buffer | a new allocation per row | | a value of the wrong kind | cannot exist | discovered mid-pass, may fail there | ## The claim to be careful with "A function you hand in is called once per value, so it is always slow" is the sentence to avoid, because it attributes the cost to the wrong thing. **Name what the callback receives.** Handed one value at a time, over a column where each value's kind must be established first, it pays dispatch on every row — that really is the cost this representation buys you. Handed the **whole column at once**, the same surface name does one pass and runs at full speed. And over a uniform typed column, many operations involve no callback at all: they are a compiled pass the engine chose in advance. So the slowness belongs to per-value dispatch over an untyped column, not to the idea of supplying a function. The same care applies in the other direction: **not every design falls back like this.** Where a declared column type is enforced, the column never becomes a holder for anything, and this cost never arises — the run stopped earlier instead. ## Second-order costs - **The result inherits the representation.** An operation over such a column usually hands back another one, so the next step pays again. The cost is not a one-off tax on the step that noticed. - **Ordering stops being one rule.** A sort over a uniform numeric column is a single comparison repeated. Over a column of mixed kinds, comparisons are per-kind, may follow rules you did not choose, or may refuse outright — and if they do not refuse, the ordering can be quietly meaningless. - **Aggregates may still produce a number.** Something that combines values kind by kind can return a plausible figure computed over a subset it chose itself, which is far worse than an error. - **Measurement misleads.** A shallow look at the column sees a reference per row and reports a small, uniform figure; the values it points at are elsewhere. The reported size and the real size are different numbers. ## What the interviewer wants to hear The distinction between *bigger* and *slower* is the whole question. A candidate who says "it uses more memory" has noticed the symptom people mention and missed the mechanism. The answer worth hearing is: the column lost the property that let one decision cover every row, so the decision now happens a million times, and it happens in the middle of a pass that also has to chase a reference to find out what it is deciding about.

Does handing in your own function always cost one call per value?
No, and this is the distinction to draw. Some surfaces hand your function a single value at a time, which pays dispatch on every row; others with similar names hand it the whole column at once and run one pass at full speed. Over a uniform typed column many operations use no supplied function at all. Ask what the function receives before predicting the cost.
Is there any operation on such a column that is not slower?
Yes — anything that only moves references rather than looking at values. Re-ordering rows, taking a subset, concatenating, or copying the column shuffles pointers and never asks what they point at. The penalty lands on operations that must interpret each value, which is most arithmetic, comparison and text work.
Why does a quick size check on such a column understate what it costs?
Because the column's own buffer holds one reference per row, and a shallow measurement reports exactly that: a small, uniform figure. Every value lives in its own allocation elsewhere, with its own overhead. A measurement that follows the references reports a very different number, and that is the one to quote.

saying these in an interview costs you the question

Says the column is only bigger, never that it is slower
Claims any function handed in is always called once per value
Blames interpreter overhead generally, never naming the per-row kind check
Expects comparison and sorting to behave as they did when numeric
Assumes every design falls back this way rather than refusing
Thinks the penalty ends with the step that produced the column

Why is a column that stores one reference per row, each of any kind, slower to compute over and not merely wider?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

What a typed pass is

What per-value dispatch actually does

The claim to be careful with

Second-order costs

What the interviewer wants to hear

▾common follow-ups3 questions

▾red flags6 flags

What a typed pass is

What per-value dispatch actually does

The claim to be careful with

Second-order costs

What the interviewer wants to hear

answer

go deeper