Pairing automated tests to stored manual case definitions by matching their titles looks free. In a case-management product, why does it break, and what does a durable identifier change?
answer
- titles are editorial, ids are not
- a rename is silent, a delete is loud
- translation and duplication both forge a match
- the identifier outlives every wording change
basics
~20 sTitles are editorial text: they get reworded, translated, duplicated and re-scoped, and none of those edits signals that coverage changed. A durable identifier survives every rename, so the pairing breaks only when the definition is actually deleted.
solid answer
~50 sTitle matching couples the join to the one field humans edit most and version least. A reword during review, a translated tree, two definitions that legitimately share a name in different folders, or a case split in two all break or misroute the match — and every one of them is invisible, because the run stays green and the repository simply shows fewer definitions as automated. A durable identifier decouples the join from the wording: rename the definition, move it, rewrite its steps, and the tie holds, because the only edit that can break it is deleting the definition. What the identifier costs is discipline — someone must put it in the code and something must check it. That check is cheap: fail the push, or the build, when a declared identifier matches no definition, and periodically list definitions nothing claims.
go deeper
Recall that a case title is prose written for people and gets edited freely, so anything that joins on it can come apart without a single test failing.
Explain the specific breakages — reword, duplicate name, translation, split — and say why a durable identifier reduces the breaking edits to exactly one: deleting the definition.
Demonstrate the reconciliation you would run in both directions, and how a push-time failure on an unresolvable identifier turns silent coverage loss into a dated, named defect.
Argue the migration: what it costs to convert a title-matched estate, why fuzzy-match output must be reviewed before it is committed, and what you require of teams before automation counts at all.
## Why title matching gets chosen It costs nothing on day one. The stored definition is called *Refund is rejected after the return window closes*; the automated test is called something close to that; a script normalises case and punctuation and joins them. No annotation, no library, no identifier to look up, and the first report looks convincing. The problem is that the join is built on **the one field humans edit most and version least** — a title is prose written for a reader, not a key written for a machine. ## The four events that break it 1. **A reword.** Someone tightens the definition's wording during review, or renames the test to match a refactored method. Nothing failed, nothing was flagged; the pair simply stops matching and the definition quietly reverts to looking manual. 2. **Duplicate titles.** Two definitions in different folders, for different products or platforms, legitimately carry the same short name. The join now has two candidates and picks by whatever tiebreak the script happens to have, so an outcome can land on the wrong record. 3. **Translation and localisation.** A tree maintained in more than one language, or a title carrying a locale-specific term, cannot be matched by an English test name at all. 4. **A split or merge.** A definition is split in two after the automation was written. The old title survives on one half, so the automated mark follows the title rather than the behaviour, and the other half looks uncovered or covered essentially at random. | Event | Title match | Durable identifier | |---|---|---| | Definition reworded | Silently unlinks | Holds | | Test method renamed | Silently unlinks | Holds, id moves with the method | | Two definitions share a name | Ambiguous, may misroute | Unambiguous | | Definition split in two | Follows the surviving wording | Explicit: reclaim or redeclare | | Definition deleted | Looks like a rename | Fails loudly on push | The last row is the important one. Under title matching, **deletion and rename are indistinguishable** — both present as *no match*. Under identifiers they are different events: a rename changes nothing, while a deletion produces an identifier nothing recognises, which is a hard, nameable failure. ## What the identifier actually changes It moves the join off editorial text and onto a handle the repository guarantees. Three consequences follow: - **Refactoring becomes free.** Renaming the test, moving it between classes or files, rewording the definition's steps — none of it touches the pairing. - **Failures become loud.** The only breaking edit left is deletion, and it surfaces as a declared identifier that resolves to nothing. - **The suite becomes queryable.** You can ask the code what it claims — scan for declarations, list the identifiers — and compare that set against the repository's set. Title matching offers no such set, only a fuzzy join you re-run and hope about. ## What it costs, and how to make it stick The cost is discipline, and it is real: someone has to look up an identifier and put it in the source, and nothing about writing a test naturally reminds them. Three mechanisms carry the discipline without a policing rota: 1. **Fail the push on an unresolvable identifier**, by name, per item, without discarding the rest of the batch. This catches deletions and copy-paste typos on the day they happen. 2. **Reconcile in both directions on a schedule.** List declared identifiers nothing matches, and definitions nothing claims. The first list is a defect; the second is a work list a human has to read, because a definition nothing claims is either honestly manual or automated by a test that forgot to declare. 3. **Detect duplicates.** The same identifier on two tests is a legitimate pattern in some suites and a copy-paste accident in most, so make it visible rather than illegal. Two tests claiming one definition means two outcomes racing to be its last recorded result. ## The migration trap Teams usually arrive here already holding a title-matched estate, and the tempting move is to run the fuzzy match once and write its output back as identifiers. That is reasonable only if the output is **reviewed before it is committed**, because the fuzzy join's ambiguous cases — exactly the duplicate titles and split definitions above — are the ones it resolves worst. Baking a bad match into an annotation converts a soft, recoverable error into a durable one that now looks authoritative and will be trusted for years.
- You inherit a suite that has always matched on titles. What is your first move?Run the fuzzy match once as a proposal, never as a commit. Review the ambiguous cases by hand — duplicate titles and split definitions are exactly what a fuzzy join resolves worst — then write the reviewed identifiers into the code and turn on a push-time check so the next unresolvable identifier fails by name instead of vanishing.
- Two automated tests end up declaring the same case definition. Is that a defect?Usually, though not always. Deliberate double coverage exists, but the common cause is a copy-pasted declaration, and the effect is two outcomes racing to be the definition's last recorded result. Make duplicates visible in reconciliation rather than rejecting them outright, and require a reason for the ones you keep.
The identifier is the ISBN on a book: the cover title can be reprinted, retitled or translated, but a catalogue joins on the number, so nothing on the cover can break the link.
saying these in an interview costs you the question
- Believes a normalised title match is as durable as an identifier
- Thinks a broken title match will show up as a failing test
- Cannot distinguish a renamed definition from a deleted one
- Writes fuzzy-match output straight into code without review
- Assumes titles are unique across the whole case tree