A result is carried through five filtering and reordering steps and must be reattached to the records it came from - what must travel with the values?

## Position is a correspondence, not an identity When two holdings have no per-row identity, the only thing relating row 7 of one to row 7 of the other is that both are seventh. That is a genuine correspondence and it is perfectly usable - as long as nothing has disturbed either side. It is not an identity, because it is a property of the holding's current arrangement rather than of the record. The distinction only matters once a pipeline has steps in it, which is to say always. ## What breaks it, and how quietly A correspondence by position is broken by any step that changes which rows are present or what order they are in: - **filtering** - one side now has fewer rows, and every position after the first dropped row refers to a different record; - **sorting** - both sides may still be the same length, which is the dangerous case, because the lengths agreeing looks like evidence; - **grouping** - the result has one row per group, and positions now index groups rather than records; - **de-duplication** - rows vanish somewhere in the middle, and nothing marks where; - **reading the two holdings from different sources**, which may simply never have agreed on order in the first place. In none of these does anything raise an error. If the two sides are the same length, the operation succeeds and produces a full-length result with values attached to the wrong records. If they are different lengths, you get a failure - which is the lucky outcome, because it is loud. ## Where identity lives: three designs, three obligations 1. **The design materialises a row-label structure beside the values.** Identity is first-class and carried by default: filter the rows and the survivors keep their labels, so a later step can bring two holdings together by identity. Your obligation is to know when an operation replaces or resets those labels, and to remember that nothing checks them for uniqueness. 2. **The design has no row-identity concept at all.** Rows are addressed by position, full stop. Identity has to be an ordinary field that you select through every step and match on explicitly at the end. Your obligation is that the field survives: a step that keeps only the measures will drop it, and it will do so without comment. 3. **The design keeps row names but discourages them** - restricted, and dropped by most operations. The obligation is the most awkward of the three, because the mechanism exists and cannot be relied on, so in practice you treat it like the second case. None of the three is wrong. What is wrong is writing code that assumes the first while running on the second. ## The two halves of what labels buy, seen here together | what you need at the end | what has to have travelled | what fails if it did not | |---|---|---| | attach a value to the right record | an identity per row, carried or explicit | full-length result, wrong rows, no error | | say what each number means | field names on the result | a rectangle nobody can check by reading | | prove the two sides still describe the same records | identity present on both sides | no evidence exists in the values either way | That last row is the one people miss. Without an identity on both sides there is nothing in the data that could confirm or deny the correspondence - not the length, not the values. The absence of evidence is the whole hazard. ## The interpretability half The same pipeline has a second failure that is slower and just as expensive. A result carried positionally moves the meaning of every offset out of the data and into whoever wrote the code. Five steps later nobody can check the result by reading it; they can only re-derive it. Names keep the mapping attached to the values, where a reviewer and the next step both see the same thing. Neither names nor identity enforce anything, but both keep meaning and correspondence from drifting away from the numbers they belong to. ## The working rule - Decide at the start of the pipeline what identifies a record, and say it out loud in the code. - If the design does not carry identity for you, carry it as an ordinary field and select it through every step. - Never treat equal length as evidence of correspondence; it is the failure mode's best disguise. - Remember that whichever carrier you use, nothing checks it for uniqueness - reattaching by identity assumes something the holding does not guarantee.