skip to content

How do you trace a requirement to its scenarios and to those scenarios' latest run results?

level: middleimportance: should knowfreq 50%

answer

  1. Three links, not two
  2. Link by identifier, never by title
  3. Generated on every run, not typed
  4. Read the matrix in both directions
  5. Linked and passing are different claims

basics

~20 s

Tag each scenario with a stable requirement identifier, then generate a matrix from those tags joined to the last run's results. Read it both ways: requirements with no scenario, scenarios with none, and requirements whose scenario failed.

solid answer

~50 s

The chain has three links: requirement to scenario, scenario to the result of the most recent run, and the whole thing read backwards to find gaps. The link itself should be a stable machine-readable identifier carried as a tag on the scenario, not the requirement's title, which breaks silently the first time someone rewords it; a tag pointing at an identifier that does not exist should fail the build. The matrix is then generated on every run rather than kept by hand, because a hand-maintained table decays from the day it is written and its decay is invisible. The value sits in the orphans: requirements with no scenario, scenarios linked to nothing, and above all requirements whose linked scenarios are failing or were not executed — those look covered in a naive matrix while the only evidence for them is red.

code

pseudocode · 17 lines
pseudocode
requirements = load_requirements()      # id -> title
scenarios    = load_scenarios()         # name -> tags
results      = load_last_run()          # name -> PASSED | FAILED | NOT_RUN

matrix = {}
for req in requirements:
    linked = [s for s in scenarios if ("req:" + req.id) in s.tags]
    matrix[req.id] = [(s.name, results.get(s.name, "NOT_RUN")) for s in linked]

unspecified = [r for r in matrix if len(matrix[r]) == 0]
demonstrated = [r for r in matrix
                if len(matrix[r]) > 0
                and all(status == "PASSED" for (_, status) in matrix[r])]
unlinked = [s.name for s in scenarios
            if not any(t.startswith("req:") for t in s.tags)]

report(unspecified, demonstrated, unlinked)

go deeper

for a junior

Know that scenarios can carry tags, and that a tag naming the requirement is what lets a report say which requirements have scenarios behind them. Be able to describe the idea even if you have not built one.

for a middle

Explain the mechanics: stable identifiers rather than titles, a build check that rejects unknown identifiers, and generation of the matrix from tags joined to the last run's results rather than a maintained table.

for a senior

Demonstrate that you read the matrix for gaps — unspecified requirements, unlinked scenarios, and requirements whose evidence is red or missing — and that you know how a hand-kept table decays invisibly between audits.

for a principal

Own how much traceability the organisation actually needs. Full requirement-level tracing is cheap to generate and expensive to keep meaningful, and it is worth arguing for it only where a regulator, partner or audit consumes the output.

### The three links Traceability in a scenario suite is a chain of three links, and interviewers are usually probing whether you know that the third one exists. 1. **Requirement → scenario.** Which scenarios claim to demonstrate this requirement? 2. **Scenario → result.** What did each of those scenarios do in the most recent run? 3. **Result → requirement, read backwards.** Given the last run, which requirements are currently demonstrated, which are demonstrated-but-failing, and which have nothing behind them at all? A traceability matrix that stops after link 1 is a coverage claim with no evidence: it says a scenario exists, not that anything passed. The reason living documentation is a better home for traceability than a hand-kept spreadsheet is precisely that link 2 is generated from a run rather than asserted by a person. ### How the link is made The workable mechanism is a **stable identifier**, carried on the scenario as a tag or a structured comment, that names the requirement: `req:PT-4417` on the scenario, `PT-4417` in whatever system holds the requirement. Three properties matter: - **Stable.** The identifier must survive renaming, re-wording and re-prioritising the requirement. Linking by requirement *title* breaks the first time a title is edited, and breaks silently. - **Machine-readable.** A human convention ("we put the ticket number in the scenario name, usually") produces a matrix full of holes. Parse it, don't eyeball it. - **Validated in the build.** A tag referencing an identifier that does not exist should fail the build, or the matrix slowly fills with links to nothing. Many-to-many is normal and should not be flattened: one requirement is usually demonstrated by several scenarios, and one scenario often touches more than one requirement. Model it as a set on both sides. ### Reading it in both directions The value is in the orphans, not the matches. **Requirements with no scenario.** Nothing demonstrates them. This is the number a stakeholder actually asks for, and it is the one a spreadsheet is worst at keeping honest. **Scenarios with no requirement.** Either the scenario is testing something nobody asked for, the requirement was withdrawn and the scenario outlived it, or the tag was forgotten. All three are worth surfacing; none of them is automatically a defect. **Requirements whose scenarios are failing or did not run.** These are the dangerous middle. A matrix that colours "has a linked scenario" green regardless of the run result will report a requirement as covered while the only evidence for it is red. Requirement coverage — the share of requirements with a *passing* linked scenario in the last run — is a different and far more useful number than the share with any linked scenario at all. Note that this is requirement coverage, not line or branch coverage of code; they answer unrelated questions and conflating them in an interview is a visible error. ### A worked example The four-person team on the parcel-tracking gateway tags each scenario with the requirement it demonstrates. Their generated matrix, published beside the specification report, shows 61 requirements: 54 with a passing scenario, 3 with a failing one, 2 whose scenarios were not executed in that run, and 2 with no scenario at all. After the caching change that produced a stale-cache read, the "status reflects the most recent depot scan" requirement moves from the passing column to the failing one — and the matrix, not a person, is what says so. The two requirements with no scenario are the honest gap the team carries into planning. ### Where it goes wrong The classic failure is a **parallel artefact**: a spreadsheet or a wiki table listing which scenarios cover which requirements, maintained by hand. Nothing in the build fails when a scenario is renamed, deleted or re-tagged, so the table decays from the day it is written, and its decay is invisible until an audit. If traceability matters enough to keep, it has to be *derived* — regenerated on every run from the tags that are actually in the repository and the results that actually came back. A second failure is over-fine linking: tagging every step, or every example row, with an identifier. The maintenance cost lands on the team every time a requirement is reorganised, and the extra resolution answers no question anyone asks. The scenario is the right unit. ### How to answer well Name the three links; insist that the second one is generated, not typed; describe reading the matrix in both directions for orphans; and state plainly that "a scenario exists" and "a scenario passed on this revision" are different claims and only one of them is evidence.

  • What do you do when a requirement is renumbered after its scenarios were already tagged?
    Keep the old identifier resolvable. Either the requirement store keeps the former identifier as an alias, or the generator carries a small redirect map that is itself reviewed. The build should fail on a tag that resolves to nothing, so a renumbering that breaks links is caught on the next run rather than showing up as a mysteriously uncovered requirement.
  • How should the matrix report a requirement whose only linked scenario is currently failing?
    As not demonstrated, visibly distinct from both covered and unspecified. A link plus a red result is evidence that the requirement is not met right now, which is more urgent than a requirement with no scenario at all. Colouring it green because a link exists is the defect this artefact exists to prevent.
  • Is tagging every step or every example row with a requirement identifier a good idea?
    No. The scenario is the right unit. Step-level or row-level tagging multiplies maintenance every time requirements are reorganised and answers no question anyone actually asks. Keep the resolution at the scenario, and split a scenario if it genuinely demonstrates two unrelated requirements.

saying these in an interview costs you the question

  • Keeps the traceability table in a hand-maintained spreadsheet
  • Links scenarios to requirements by title rather than identifier
  • Counts a requirement as covered when its scenario is failing
  • Reports only requirement-to-scenario and never the run result
  • Confuses requirement coverage with line or branch coverage of code
  • Never checks for scenarios linked to no requirement at all

context

resolving links…