Setting an identifier column as a ten-million-row table's row labels raised its memory. What was allocated that the default labelling never allocated?
answer
- the default is a rule, not values
- arbitrary labels have no rule
- one entry per row, materialised
- a lookup structure beside them
- text labels mean a reference per cell
basics
~20 sA full-length sequence of label values, plus whatever structure the design builds beside it to make lookup by label fast. The default consecutive labelling is usually held as a rule — a start, a step and a length — so it materialises nothing.
solid answer
~40 sThe default labelling in several designs is not stored at all: it is a description of a range, so ten million rows cost a handful of bytes. Setting a labelling from a column replaces that description with real values — one entry per row, and if the identifiers are text, the entry is a reference to a separately allocated string rather than a packed number. On top of that, the design builds a lookup structure over the labels so that finding a row by name does not mean walking ten million entries; that structure is a second full-length allocation, built eagerly or on first use depending on the design, and rebuilt when rows change. Depending on the operation you may also still be holding the original column, so the same values are resident twice.
go deeper
Know that a labelling set from a column stores real values for every row, while the default numbering usually does not store anything at all.
Explain both allocations — the full-length sequence of label values and the lookup structure over it — and why text identifiers cost more than packed numbers.
Judge the trade in context: count the lookups the step actually performs against a full-length allocation, and notice when the same values are resident twice.
State the cost conditionally, since designs without row identity pay nothing here and scan instead, and write any team guidance so it holds in both.
## Two labellings, two completely different costs The word *labelling* covers two things whose cost models have almost nothing in common. **The default consecutive labelling.** When a table is built and nobody says otherwise, rows are labelled 0, 1, 2 and so on. Several designs notice that such a labelling is describable rather than storable and keep it as a **rule** — a start, a step and a length — so ten million rows cost tens of bytes. Nothing is materialised; a label is computed when asked for. **A labelling set from a column.** Now the labels are arbitrary values, so there is no rule to describe them. The design materialises a full-length sequence, one entry per row, in whatever representation those values need. That difference is the whole answer to the question, and it is why "the row labels are free, they are just the row numbers" is true of exactly one case and false of the case you just created. ## What the change actually allocated 1. **A full-length sequence of label values.** Ten million entries. If the identifiers are fixed-width numbers, that is the packed width times the row count. If they are text, most designs hold a reference per cell pointing at a separately allocated string — so you pay a pointer per row *plus* the strings themselves, which is comfortably several times the packed-number case. 2. **A lookup structure over those labels.** The reason to have labels at all is to find a row by name without walking the table, and that requires something hash-like beside the values. It is another full-length structure. Designs differ in *when* they build it: some build it as part of the assignment, some build it lazily on the first lookup, so the memory step may appear a moment after the line you suspect. 3. **Possibly a second copy of the values.** Some operations move the column out of the table and into the labelling; others leave the column in place. Where the column stays, the same ten million identifiers are resident twice. 4. **Bookkeeping about the labels.** Designs commonly cache cheap properties — whether the labels are in ascending order, whether they are known unique — because those decide which lookup path is available. It is small, but it is state that has to be invalidated when rows change. ## What you bought in exchange | operation | with the default rule-based labelling | with a labelling set from a column | |---|---|---| | find the row named ACC-1042 | not expressible by label; test a column over every row | resolved through the lookup structure | | select a span of ordered identifiers | not expressible by label | two boundary searches, when the labels are ordered | | reach row 5,000,000 | direct, by offset | direct, by offset, unchanged | | memory for ten million rows | a rule, effectively nothing | one sequence of values plus a lookup structure | The trade is legible: you converted bytes into named access. Whether that is a good trade depends entirely on how many lookups by name the step actually performs. A pipeline that sets a labelling, performs one retrieval and then reduces the table has paid a full-length allocation for a single lookup it could have done by testing a column once. ## Designs that pay nothing here A design with **no row-identity concept** never faces this cost and never gets the benefit: identity is an ordinary column, and finding an entity means testing that column, which is a pass over the values and allocates nothing lasting. A **uniform-type rectangle** — one buffer of one representation, addressed only by position — has no labels to allocate either. So the honest statement of the cost is conditional: *in designs that materialise a row-label set, setting one from a column allocates a full-length sequence plus a lookup structure; in designs without row identity, the same work is a repeated scan and the memory stays flat.* ## The diagnostic habit When memory moves after a line that looks like bookkeeping, ask three questions: what is now materialised that was previously a rule; is the value representation packed or a reference per cell; and is anything holding the same values twice. On this particular change, all three can be yes at once, which is why a step that appears to rename something can cost more than the table it was applied to.
- Why does the memory sometimes rise a moment after the assignment rather than at it?Because the lookup structure is not always built eagerly. Several designs materialise the label sequence at the assignment and only build the structure over it on the first retrieval by label, so the second step of the cost lands at the first lookup. Timing a memory jump to a line is therefore unreliable evidence about which line caused it.
- The identifiers are text and most of them repeat across rows. Does that change the cost?It can, sharply. A representation that stores each distinct value once and a small code per row collapses the per-row cost for a column of few distinct values. Whether a labelling can use such a representation varies by design, so the honest answer is that repeated text is an opportunity, not a guarantee.
- When is paying for a labelling clearly worth it?When the step performs many retrievals by name over the same holding, or repeatedly selects spans of ordered identifiers. The allocation is paid once and amortised over every lookup. One retrieval does not amortise anything, and a single pass testing a column would have been cheaper in both time and memory.
saying these in an interview costs you the question
- Says row labels are always free because they are just row numbers
- Thinks the label values are stored once per column
- Ignores the lookup structure built beside the label values
- Assumes the source column is always removed from the table
- Believes setting a labelling changes the columns' representations