Steps in your team's pipeline come from tools that assume different layouts — which arrangement do you commit to as the interchange form, and what do you accept by choosing it?

## What is actually being decided The per-script question is *which arrangement does this next run of operations read.* The standing question is different: **what arrangement do datasets have when they pass between the parts of the system that different people own.** It is an interface decision, made before the pipeline is written, and it is the sort of thing a lead owns because its cost is paid by everyone else, in small amounts, for a long time. There is no universally correct answer. The commitment is worth making anyway, because the alternative is not neutrality — it is each step inheriting whatever its own tool assumes, which is how a pipeline ends up converting at twenty points. ## The two commitments, and what each one costs | | Commit to the long layout | Commit to the wide layout | |---|---|---| | Interface stability | fixed set of columns; a new measure is rows | column set comes from the data and can move between runs | | Cross-measure work | every consumer converts for itself | direct, no conversion | | Identifying columns | repeated once per measurement | written once per subject | | Type discipline | one value column, so one representation for every measurement | each measure keeps its own | | Adding a measure | invisible to consumers | a schema change everyone sees | The row that usually decides it is the first one. A wide interchange form makes **next month's column set a function of last month's data** — a distinct value that has never appeared before becomes a column nobody wrote, and a value that stops appearing takes its column away. That is manageable inside one script and is an operational risk across a team, because the consumer that breaks is not the code that changed. The row that argues the other way is the second. If the mass of the work is measure-against-measure, a long interchange form means every one of those consumers performs the same conversion, which is duplicated work in the literal sense — the same rearrangement written several times, and several chances to write it differently. ## How to actually decide 1. **Count the consumers, not the steps.** How many places take the data across the interface, and how many of them are doing measure-against-measure work? If the answer is *most*, the wide form earns its schema risk; if it is *one*, that one converts and everybody else gets a stable interface. 2. **Ask where the set of measures comes from.** If measures arrive from the outside world and the set grows, a fixed interface is worth a lot, and that argues for the long form. If the set of measures is defined by your own model and changes only when someone edits it, the wide form's schema risk is much smaller. 3. **Place the boundaries deliberately.** The commitment is only half the decision; the other half is naming where conversions happen — ideally at the edges, one on the way in and one before the analytical stretch, each with the operations it serves written next to it. 4. **Decide what a stable wide interface would require.** If you commit to the wide form, the honest version is not *whatever columns come out* but a declared set of measures, with anything outside it handled explicitly. That turns a data-derived schema into a stated one, and it is the work you are signing up for. ## The cost nobody prices Two conventions in one team is worse than either commitment. It shows up as two implementations of the same rearrangement that disagree at the edges, a reviewer who cannot tell whether a step's input is in the agreed form, and a slow drift where new work adopts whichever arrangement the newest tool assumed. The value of committing is mostly the value of everyone knowing the answer without asking. ## The claim that must not travel The most durable part of the decision is a rule rather than a layout: **a layout claim from one ecosystem does not get carried into another without naming the mechanism it depends on.** The tools in this family genuinely disagree — about which arrangement they assume, about whether operands are matched by label or by position, about what happens to cells a conversion had to create, about whether a supplied reduction sees one value or all of them. A team that treats one tool's behaviour as the behaviour will ship a correct-looking step that is wrong in the other half of the pipeline, and that class of defect is not caught by a schema check. So the artefact of this decision is short: the committed interchange arrangement, the named boundaries where conversion happens, the declared set of measures if the wide form won, and the standing rule about carrying claims across. One page, and it settles arguments that would otherwise be re-held in every review. ## What an interviewer is listening for Not a preference. They want to hear the tradeoff priced — stable interface against direct cross-measure work — the decision tied to where the consumers actually are, the boundaries placed explicitly, and an awareness that the wide form's convenience is bought with a schema that the data writes.