A job keeps ten rows out of each million-row table it reads, yet memory never falls - what should you suspect about the kept results?
answer
- tiny results, large footprint
- one step per source, never down
- a reference is to the whole buffer
- materialise at the boundary
basics
~20 sSuspect the kept results are windows onto the originals' bytes. A window holds a reference to the whole buffer, so ten visible rows keep a million resident, and nothing is released until the kept rows are materialised into storage of their own.
solid answer
~50 sA result that shares the original's storage holds a reference to **the whole buffer**, not to the part you can see. So ten visible rows can keep a million rows resident, and the footprint does not fall when the source goes out of scope - the window is still referring to it. The symptom is characteristic: the row counts in the program are tiny, the memory is not, and it grows once per iteration rather than continuously within one. The fix is to **materialise deliberately** - copy the kept rows into storage of their own at the boundary where you stop needing the source, so the last reference to the big buffer goes away. If the selection had been a gather, the result would already own its bytes and the source would be releasable once nothing else referred to it, which is why the symptom itself points at the kind of selection that was used.
go deeper
Take away the fact itself: a small result can keep a large one alive, so row counts are not a measure of memory.
Explain the mechanism - the result refers to the whole buffer and a buffer is released only when nothing refers to it, so partial release is not something it can do.
Show the diagnosis: tiny results with a footprint that steps once per source, confirmed by materialising the kept rows and watching the step disappear, with the fix placed at a boundary rather than everywhere.
Make it a contract: a function that returns a selection out of something large it read itself owes its caller a result that does not pin an invisible source, and a review can ask for that without measuring anything.
## The shape of the symptom This is one of the few memory problems that announces itself clearly if you know what you are looking at. The signature is: - the program holds **tiny** results - a handful of rows per iteration; - the footprint is **large** and roughly a multiple of the source, not of the results; - it steps up **once per source processed** and does not come back down; - nothing is obviously leaking, in the sense that every variable the code holds is small. That combination is what retention through a shared result looks like. The code is holding exactly what it meant to hold. What it did not mean to hold is everything those small results are still attached to. ## Why a small window pins a large buffer A **window onto the same bytes** holds no values of its own. What it holds is a reference to the original's storage plus a description of which part of it to read - where to start, how many, how far apart. The reference is to the buffer, and a buffer is released only when nothing refers to it at all. It has no way to release the parts of itself nobody is looking at, because parts of a buffer are not separately owned. So the accounting is not *ten rows' worth of bytes*. It is *one whole source, kept alive by a ten-row description of it*. Keep one such result per file across a thousand files and the process holds a thousand sources, while every row count in the program says it is holding ten thousand rows. The mirror-image claim is the one worth carrying into the interview, because it is also often stated wrongly: **keeping ten rows out of a million releases the rest only if the result owns its bytes.** A gather - an arbitrary list of positions, a condition column - allocated fresh storage for the surviving values, and the original is then collectable once nothing else refers to it. A window does not release anything, and the footprint stays flat until the window itself is materialised or goes away. ## Confirming it rather than guessing 1. **Ask what shape the selection was.** If the rows were picked by a condition or by a list of positions, the values were gathered and this diagnosis is wrong - look elsewhere. If they were a contiguous run or a regular step, sharing was possible and this is the first hypothesis. 2. **Check what is still referred to.** Retention means a live reference, so trace what the kept result is attached to rather than how big it prints. 3. **Test the hypothesis directly.** Materialise the kept rows into storage of their own before storing them, and see whether the step-per-source disappears. If it does, the diagnosis is confirmed; if it does not, something else holds the source. ## The fix, and where to put it The repair is **deliberate materialisation at a boundary**: at the point where the code stops needing the source, copy the rows it is keeping into storage of their own so the last reference to the big buffer goes away. The important word is *boundary*. Materialising everywhere turns every step into an allocation and gives up the cheapness that made sharing worth having; materialising nowhere is the bug. The natural boundaries are: - **A function that returns a small selection out of something large it read itself.** Its caller has no way to know what the returned object is attached to, so the function should not hand out a window onto a source only it could see. - **Anything that outlives the loop iteration** - an accumulator, a list of per-file results, a cache. - **A result that crosses into long-lived state** while the source was transient. A result that is consumed immediately and dropped inside the same iteration does not need any of this. The cost of a window is retention, and retention only matters when something is retained. ## The design variation to keep in mind How much is pinned depends on what the design shares. Where a selection is a window on one buffer, that buffer is what stays resident. Under designs that are immutable by default and return a new handle sharing every buffer the step did not touch, the small result can be attached to whole untouched columns rather than to one buffer - the same retention with a different accounting. Under designs that duplicate eagerly, the problem does not arise at all, and under designs that defer the duplicate until something is written, the result behaves like a shared one until a modification forces the allocation. So the diagnosis generalises but the arithmetic does not, and the fix - materialise at the boundary - is the same in all of them. ## How to answer this in an interview Name the mechanism first: a window holds the whole buffer alive, so the footprint tracks the sources, not the results. Then show that you know when the diagnosis does *not* apply - a gathered result already owns its bytes - and finish with the boundary-shaped fix rather than a blanket rule to copy everything.
- Why does the footprint not fall when the source variable goes out of scope?Because the kept result is still referring to the source's buffer. Leaving scope removes one reference, not all of them, and a buffer is released only when nothing refers to it. The small result is the remaining reference, and it refers to the whole buffer rather than to the part it shows.
- Would the same symptom appear if the rows had been picked by a condition?No. A condition keeps an irregular set of rows, so the values had to be gathered into freshly allocated storage; that result owns its bytes and the source becomes collectable once nothing else refers to it. Seeing the symptom therefore points back at a selection that could be described as a stride.
- Why not just materialise every intermediate result to be safe?Because it pays an allocation proportional to every step, including the many that are consumed and dropped immediately, which is exactly the cost sharing exists to avoid. Retention only matters where something is retained, so materialise where a result outlives its source - a return value, an accumulator, a cache - and nowhere else.
Keeping one page of a library book by leaving a bookmark in it: the bookmark is tiny, but the book cannot go back on the shelf while you have it out, and your shelf fills up with books you are each using one page of. Photocopying the page is the move - once you hold the copy, the book goes back, and it is the copy, not the discipline of remembering to return things, that makes the shelf empty again.
saying these in an interview costs you the question
- Assumes a ten-row result can only cost ten rows of memory
- Says the source is freed as soon as it leaves scope
- Copies every intermediate to be safe, losing the point of sharing
- Concludes a gathered result also pins its source
- Blames the garbage collector rather than a live reference