Doubling two million numbers by looping over records and assigning each result back: what is paid once per record?
answer
- four fixed costs, paid n times
- the column has one stored representation
- values handed back wrapped, not raw
- a whole record rebuilt per row
- the result container regrows
basics
~20 sFour fixed overheads move from once-per-column to once-per-record: a run-time type decision per value, each value handed back as a wrapped language object, a fresh record built to reach its fields, and a result container reallocated as it grows.
solid answer
~50 sThe loop is not slow because multiplication is slow. It is slow because four fixed overheads that a whole-column form pays once are paid two million times. First, a run-time decision about what the two values are and which operation applies; a column commits every value to one stored representation, so that decision can be made once before the pass. Second, each value is handed back as a full language object with its own header instead of staying raw bytes in a packed buffer, and the answer is wrapped again on the way back. Third, if the surface hands back whole records, one is assembled per row even when the body reads two fields. Fourth, the result container is reallocated and copied as it grows. Saying the operation once for the whole column, working column-wise, collapses all four into one decision, one pass over packed bytes and one allocation.
code
pseudocode · 10 lines# per-record: every cost below is paid once for each of n records
out = empty_growable()
for i in 0 .. n-1:
v = read_value(column, i) # handed back as a wrapped language object
twov = v * 2 # run-time decision: what are these two things?
push(out, twov) # container reallocated and copied as it grows
# whole-column: every cost above is paid once for the whole column
out = column * 2 # one decision, one pass over the packed bytes,
# one result allocated at its final lengthgo deeper
Be able to say that the same fixed work is repeated once per record: a type decision, a wrapped value, a rebuilt record, a regrowing result. Naming two of the four convincingly is enough at this stage.
Explain why one stored representation for the whole column is what lets the decision be hoisted out of the loop, and price the overheads against each other rather than reciting them as a flat list.
Show how you would establish which overhead dominates on real data instead of asserting a ranking, and say which of the four grows worse than linearly as the row count rises.
Consider what rule the team should carry: where per-record loops are banned outright, where they are tolerated for clarity, and what readability is worth against the runtime the column-wise style buys.
## What is actually being compared A loop over records and a whole-column expression compute the same numbers. The multiplication itself is the same multiplication. What differs is everything wrapped around it: a small set of fixed overheads that the record loop pays **once per record** and the column-wise form - saying the operation once for the whole column rather than once per row - pays **once for the whole column**. At two million records that is two million payments of each overhead against one. Three terms first, because the rest rests on them: - **A packed typed buffer** is values of one representation laid end to end and addressed only by position. A numeric column's values normally live in one of these. - **A boxed value** is a value held as a full language object, with its own header and a pointer to it, rather than as raw bytes inside such a buffer. - **The column's stored representation** is the single representation every value in that column shares, and the fixed number of bytes it therefore costs per row. ## The four overheads 1. **A run-time type decision, once per value.** In a general loop nothing promises what the two operands are, so before the multiplication can happen the program has to work out what it is holding and which concrete operation applies. A column makes exactly that promise: every value in it shares one stored representation, so the decision can be made once, before the pass begins, and never again. 2. **Wrapping, paid twice per value.** Reading a value out of a packed buffer in order to hand it to your loop body means producing a language object for it - allocating a header, filling it, and later collecting it. The arithmetic result is then wrapped again on the way back. The bytes were already there; the wrapping is pure accounting. 3. **A record rebuilt, once per row.** Columns are stored separately, so a record does not exist anywhere in memory until something assembles one. A surface that hands records back one at a time must gather one value from every column, wrap each, and build a container, on every iteration, whether the body reads two fields or forty. 4. **A result container regrown.** Results appended inside the loop go into a container that does not know its final size, so it is periodically reallocated and its contents copied across. In the mild form this amortises away. In the severe form, where each step builds a whole new result rather than extending one, the copying grows with the work already done - and that is the one overhead here whose share of the total gets worse as the input gets bigger. ## What the column-wise form pays instead | overhead | record loop | whole-column form | |---|---|---| | type decision | once per value | once for the column, before the pass | | wrapping and unwrapping | twice per value | never; the pass walks raw bytes | | record assembly | once per row | never; columns are read as columns | | result allocation | as the container regrows | once, at the final length | | the arithmetic itself | once per value | once per value | The last row is the point. The arithmetic did not get faster. It is the same work with the accounting removed from around it. ## What varies between designs, and what does not - **Where the host language is compiled and stores values unwrapped**, the first two overheads largely disappear: the operation is resolved before the loop runs and no object is built per value. What remains is the per-record assembly and whatever the accumulation costs, so the gap narrows considerably and a cost ranking carried over from one ecosystem does not transfer to another. - **The win is not parallelism.** Some engines split a column-wise pass across threads, and that is a genuine additional multiplier. The ordinary gain described here happens on one core and comes entirely from removing per-value dispatch and unwrapping. Crediting it to cores is the commonest wrong explanation offered for the right observation. - **The loop does not vanish, it moves.** Something still visits every value. It now does so inside compiled code where the representation is already known and the values are raw, which is exactly why the visit is cheap. - **It is not unconditional.** The whole-column call has its own fixed setup - checking arguments, resolving the operation, allocating the result - which is negligible spread over two million values and can exceed the entire loop it replaced over twenty. ## Answering it in an interview Name the overheads rather than gesturing at overhead in general, and say which you would expect to lead: on a wide table the per-record assembly, on a narrow one the wrapping and the type decision, and if the timing curve bends upward as the input grows, the accumulation ahead of all of them. Then say you would measure it rather than assert it.
- If the host language were compiled and stored the values unwrapped, which of these overheads would remain?The type decision and the wrapping largely vanish: the operation is resolved before the loop runs and no object is built per value. What remains is any per-record assembly the surface performs and whatever the accumulation costs, plus the fact that the loop still issues one operation per value rather than one over a whole buffer. The gap narrows; it does not close.
- Does expressing it column-wise remove the loop?No, it moves it. The library still walks every value. The walk now happens inside compiled code where the representation is already known and the values are raw bytes, so the per-value overheads are not paid. Saying the loop was eliminated is wrong; saying the dispatch moved from once per value to once per column is right.
- Why does the column's stored representation matter to the first overhead?Because a column commits every value to one representation, the library can decide once, before the pass, which concrete operation applies and how wide each value is. A general loop carries no such promise: any pair could be anything, so the decision has to be repeated for every value.
saying these in an interview costs you the question
- Says the loop is slow because the arithmetic itself is slow.
- Says the whole-column form is faster because it uses all the cores.
- Claims the loop disappears rather than moving into compiled code.
- Treats the column-wise rewrite as faster at any column length.
- Cannot name a single per-record cost beyond the word overhead.