skip to content

Why must a case's identity in a result model be stable across runs rather than its display name or its position?

level: seniorimportance: should knowfreq 45%

answer

  1. History is joined on something
  2. Renaming should not erase a past
  3. Position shifts when a case is inserted
  4. Assign the key, never derive it
  5. The display name is a label

basics

~20 s

History is joined on identity. If a case is identified by its display name or its position in a file, rewording a sentence or inserting a case above it destroys the past outcomes and creates a brand-new case carrying none.

solid answer

~40 s

Identity is the key every downstream reader joins on: previous outcomes, an accepted-failure entry, attached evidence, a link in a review. Two tempting identities are unstable — the human-readable title, which changes the first time someone improves the wording, and the ordinal position, which changes the moment a case is inserted above it. Either change makes one case look deleted and another look new, so the suite silently forgets its history and "is this failure new?" stops being answerable. A durable identity is an **explicit key assigned once and stored in the case**: unchanged by rewording, reordering or moving the case to another file, and unique across the whole suite so results merged from several parallel test workers cannot collide. The display name stays as a label — free to change, never joined on.

code

yaml · 9 lines
yaml
case:
  id:    "checkout.expired-card.declines"   # assigned once, never rewritten
  title: "An expired card is declined at checkout"   # free to reword any time
  owner: "payments"

# joining this run against history
# for each entry in this_run.cases:
#     past = history.lookup(entry.id)        # joins on id, never on title
#     entry.is_new_failure = entry.status == FAILED and past.last_status != FAILED

go deeper

for a junior

Know that a case has a name a human reads and an identity a machine joins on, and that the two do not have to be the same value. Recognise that renaming something should not lose what happened to it before.

for a middle

Explain what history is joined on and why a reworded title or an inserted case can erase it. Be able to list candidate identity schemes and say which of them survive a rename, a reorder and a move to another file.

for a senior

Show the downstream damage you have lived with: triage that cannot separate new failures from old, accepted-failure entries that quietly stop matching, orphaned evidence, merged output that collides. Describe how you would migrate a suite that never had stable identities.

for a principal

Treat identity as a long-lived contract with everything that reads the suite's output. Decide when a case's behaviour has changed enough to deserve a new identity rather than a rewritten one, and make that an explicit, reviewable decision rather than a side effect.

## Identity is the join key for everything downstream A run's results are only interesting beside other runs' results. "Is this failure new?", "has this case been failing for a week?", "did the fix actually work?" — every one of those questions is a join between this run and earlier ones, and a join needs a key. Whatever field you join on **is** the case's identity, whether you chose it deliberately or inherited it by accident. Two fields get used as identity by accident, and both change for reasons that have nothing to do with what the case checks: - **The display name.** The sentence describing the case. It gets reworded the first time someone improves the phrasing, fixes a typo, or turns jargon into English. - **The position.** The case's index inside the file that declares it, or the order the runner happened to schedule it in. It changes the moment a case is inserted above it, or two parallel test workers report in a different order. When either changes, the join finds nothing. One case appears to have vanished along with its entire history, and a brand-new case appears carrying none. Nothing failed, nothing was deleted, and the suite has silently forgotten a month of evidence. ## What breaks, concretely | Identity scheme | Survives a reword | Survives an insertion above | Survives a move to another file | Safe to merge across workers | |---|---|---|---|---| | Display name | No | Yes | Yes | Only if names never repeat | | File plus position | Yes | No | No | Yes | | Path built from container names | Yes | Yes | No | Yes | | Assigned key stored in the case | Yes | Yes | Yes | Yes | The damage is not evenly distributed. Losing the join breaks four things at once: 1. **New-versus-old triage.** Without history, every failure looks new, so a reader spends the same urgency on a long-accepted failure as on a fresh regression — and stops trusting the distinction entirely. 2. **Accepted-failure entries.** An entry that points at a title stops matching when the title changes, and a case everyone believed was tracked starts failing loudly with no explanation attached. 3. **Evidence lookup.** Captures from earlier runs are addressed by the case they explain. Change the address and they are orphaned; they still exist, and nothing can find them. 4. **Merged output.** Results from several parallel test workers are combined by identity. A display name that repeats in two files merges two different cases into one row, and the row's history is a blend of both. ## What a durable identity looks like The properties, ordered by how often they are violated: - **Assigned, not derived.** Written once into the case as data, never computed from prose. Anything computed from the title *is* the title, hash or not. - **Opaque to wording.** Short and stable, and not required to read well — the display name carries readability, and it is free to change as often as anyone likes. - **Unique suite-wide**, not merely within its own file, so merged output cannot collide. - **Independent of location**, so moving a case between files or regrouping a suite costs nothing. - **Never rewritten.** If a case's behaviour changes enough that its old history is misleading, that is a *new* case with a new key and a retired predecessor — an explicit decision someone makes, not a side effect of an edit. ## Migrating a suite that never had one Adopting stable identity does not require throwing away what you have. Generate a key for every existing case once, derived from whatever the current identity happens to be; store it in the case as data; keep a one-time map from the old value to the new key; rewrite the stored history through that map; then switch every reader to join on the key alone. Delete the map afterwards — keeping it alive means the old value still means something, which is precisely the condition you were trying to remove. The discipline that keeps it working afterwards is small and cheap. A review that changes a display name must not touch the key. A review that does change a key has to say, in words, that it is deliberately starting a new case's history — which is a sentence people are reluctant to write, and that reluctance is the point.

  • A case is moved from one suite file to another with no change to its steps. What should its recorded identity do?
    Nothing — it should read identically before and after, because nothing about what the case checks has changed. If the identity moved, it was encoding location. That is the cheapest test of an identity scheme: reorganising a suite is a common, low-risk refactor, and it must never cost the suite its history.
  • How do you adopt stable identities in a suite that has always joined on titles, without losing the history you already have?
    Assign a key to every existing case once, store it in the case as data, and keep a one-time map from the old title to the new key. Rewrite the stored history through that map, switch every reader to join on the key alone, then delete the map. Keeping the map alive means the title still means something, which recreates the problem you were removing.

A book's catalogue number stays with it through every re-shelving and every new dust jacket; file by the title on the cover instead, and a reprint with a reworded subtitle becomes a book the library has never seen before.

saying these in an interview costs you the question

  • Treats the display name as the case's identity
  • Assumes reordering cases is harmless to history
  • Rewrites a case's key whenever the wording improves
  • Uses a position inside a file as identity
  • Cannot tell a new failure from a long-standing one