The same conversion to a plain positional rectangle cost nothing on one table and doubled peak memory on another — why?

## "Is the conversion expensive?" has no single answer The conversion from a labelled table to a plain positional rectangle is one line of code with two completely different cost profiles behind it. In one case it hands back a **window over bytes that already exist** and allocates nothing at all. In the other it **allocates a new rectangle and copies every value into it**, and while that copy runs the source table and the destination rectangle are both fully resident. The difference is not how big the data is. It is how the table was already holding its fields. ## Three storage designs, three outcomes | how the table holds a field | can the conversion return a window? | why | | --- | --- | --- | | one contiguous run per field | no | there are as many runs as fields; a rectangle is one buffer, so one has to be allocated and filled | | an ordered sequence of separately allocated chunks | no | there is no single buffer to point at, even for one field | | one in-memory slab per representation | only when the whole table is one representation | that slab already *is* a rectangle of that representation; anything else means more than one slab | The third row is where the belief "the conversion is free, it just hands you the buffer underneath" comes from. It is a true statement about a table that happens to be entirely one representation held in one slab, and a false statement about every other table — including the same table after somebody adds a field of a different representation, which can split it into two slabs and turn yesterday's free conversion into today's full copy. ## Why the copy shows up as a doubled peak During a copying conversion there is a moment when both holdings are alive at once: 1. The destination rectangle is allocated at full size, before a single value has moved into it. 2. Values are copied in, while the source is still needed to read from. 3. Only after the copy completes can the source be released — and only if nothing else still refers to it. So the peak is roughly **the source plus the result**, not the larger of the two. That is also why the doubling is invisible in an average and obvious in a peak: by the time anyone measures afterwards, one of the two holdings is gone. Two things make it worse than a straight doubling: - **A text field in the table.** The one representation covering the whole result then has to be one that can hold text, so the destination is not the same size as the source's values — it is larger, sometimes considerably. - **Something still holding the source.** If the table is referenced elsewhere, step 3 never happens and the peak becomes the new steady level. ## What a window changes besides cost A window and a freshly allocated rectangle differ in one more way that matters at a boundary: a window shares storage with the table it came from, so the two are not independent, while a copy leaves two unrelated holdings. Code that is correct against a copying conversion and subtly wrong against a window-returning one is a real hazard, and it is one you cannot see in the source line, because the source line reads identically in both cases. What does **not** differ is what crosses the boundary. The field names and any per-row key the table carried are absent from the rectangle either way. Whether bytes were copied is a cost question; what the rectangle knows about your data is the same answer in both. ## Predicting it before you run it - Ask whether every field shares one representation. If not, assume a copy. - Ask how the table was assembled. Built up by adding fields one at a time tends to leave more separate runs or more slabs behind than read in one shot. - Ask how much of the table you actually need in the rectangle. Converting a selection of fields, or a slice of rows, bounds the allocation by what you asked for rather than by the whole table. The conversion still copies; the copy is just smaller. - Ask what is still holding a reference to the source once the rectangle exists. ## The claim to avoid making Both universal statements are wrong. "The conversion is cheap, it just exposes the buffer underneath" is one design's behaviour generalised to the class. "The conversion always allocates and copies" is the opposite over-correction, and it will make you avoid a conversion that costs nothing. The accurate sentence carries its own precondition: **the conversion is a window only when the table is already one representation in one slab; otherwise it allocates and copies, and a single text field sets the representation for everything that lands in the result.** A candidate who states it that way has told you they have met more than one of these designs, which is the signal the question is really testing for.