skip to content

Which traceability findings are honest noise rather than plan defects, and how do you scope the analysis so the list stays actionable?

level: middleimportance: should knowfreq 34%

answer

  1. Not every unjoined end is a defect
  2. Some statements are verified without a case
  3. Ask whether the absence changes anything
  4. Narrow the population before running it
  5. Record exceptions with a review date

basics

~20 s

Cases written for build health or conventions, statements verified by review, and requirements not yet in scope are noise rather than defects. Scope it to the current release and record each exception with a reason and a review date.

solid answer

~50 s

Not every unjoined end is a defect. Cases that exist for build health, environment sanity or an engineering convention were never meant to verify a stated requirement. Statements verified by review or inspection have their evidence somewhere other than a case, and requirements planned for a later increment have no case because they have no code. Reporting those every week teaches people to dismiss the whole report. The dividing question is not *is a link missing* but **does the absence change what anyone does**. Scope it accordingly: fix the population to the release in hand, mark which statements need a case at all, record each exception once with a reason and a review date, and route findings to whoever can close them. Then watch the dismissal rate — if most entries are waved away, the filter is wrong, not the reader.

code

yaml · 15 lines
yaml
analysis:
  population: requirements in current release scope
  skip:
    - verificationMethod: review
    - verificationMethod: inspection
    - status: deferred
  caseKinds_not_expected_to_link:
    - build-health
    - environment-sanity
    - convention-check
  exceptions:
    - id: REQ-4120
      reason: verified by design review, evidence held with the review record
      reviewOn: next-release-planning
  route: finding -> owner of the requirement or of the case

go deeper

for a junior

Know that some test cases legitimately have no requirement behind them, such as checks that exist to keep the build and the environment healthy, and that finding one is not automatically a problem.

for a middle

Be able to sort findings into noise and defects and justify the split, and to explain how narrowing the population and recording exceptions turns an unread export into a list somebody acts on.

for a senior

Show that you measure the report itself. Describe what a high dismissal rate tells you about the filter, and how you route findings so each has a name against it rather than being published to everyone.

for a principal

Own the standing question of how much of this analysis a product warrants, who pays for the upkeep, and how you keep the exception list from quietly becoming the largest part of the record.

## Why the raw output is unreadable Run gap and orphan analysis over a whole product's links for the first time and it produces hundreds of findings. Most teams read that list once. The reason is not that the analysis is wrong; it is that it reports every unjoined end as though every unjoined end were a defect, and a large share of them are simply how the work is organised. A finding list that is mostly noise trains people to dismiss the whole thing, which costs more than not running it — the real gaps are now hidden inside a report with a reputation. So the skill is separating the two populations and then shrinking what the analysis is pointed at. ## Findings that are honest noise - **Cases that never had a requirement to serve.** Build-health checks, environment sanity checks, cases that enforce an engineering convention. They exist for the team, not for a stated behaviour. - **Statements verified by something other than a test case.** A design constraint the build enforces, a review, an inspection, a documented calculation. The verification exists; it is not a case. - **Requirements that are not in scope yet.** Something planned for a later increment has no case because it has no code. Reporting it as a gap every week is noise by construction. - **Retired statements whose cases are still present** during a cleanup window, and duplicated statements of the same behaviour where only one carries the link. - **Exploratory findings recorded as cases** for record-keeping, which were never intended as repeatable verification. ## Findings that are always real - A requirement inside the current scope with nothing verifying it, by any method. - A case whose requirement was deleted, so it now asserts an expectation nobody has agreed to. - A defect that no case reproduces. - A link whose case has never produced a result. The dividing question is not "is this link missing?" but **"does the absence of this link change what anyone does?"** If nothing changes, it is noise, and reporting it is a cost with no return. ## Scoping so the list gets read 1. **Fix the population before you run it.** Analyse the requirements in scope for the release or the change in hand, not everything the product has ever had. The list should be sized so one person can read all of it in a sitting. 2. **Classify statements by whether they require a case at all.** A statement whose verification method is recorded as review or inspection is not a candidate for a gap finding, and should not be counted as one. 3. **Record each exception once, with a reason and a review date.** An exception with no review date is a permanent hole with paperwork; an exception list that is never re-read becomes the place findings go to be forgotten. 4. **Route findings to the person who can close them** rather than publishing one list to everyone. A gap belongs to whoever owns the requirement; an orphan belongs to whoever owns the case. 5. **Run it where the change is.** Findings against untouched areas were there last month and will be there next month; findings against what just moved are the ones with a decision attached. ## Measuring the report, not just the product The report has a quality of its own and it is measurable. Track the share of findings that get dismissed rather than acted on. If most of a list is dismissed, the filter is wrong — not the reader. Two dismissal patterns tell you what to fix: | Pattern | What it means | The fix | |---|---|---| | The same items dismissed every run | Exceptions are not being recorded | Record the exception with a reason and a review date | | Many different items dismissed once each | The population is too wide | Narrow to the current scope or the changed area | The end state is a short list where every entry has a name against it and a plausible next action, and where the volume rises when something genuinely changes. That list gets read. A weekly export of every unjoined end in the product does not, and the analysis is worth exactly what somebody does with it.

  • How do you keep an exception list from becoming a place findings go to be forgotten?
    Give every exception a reason and a review date, and re-read the list on a fixed rhythm rather than when somebody remembers. An exception with no date is a permanent hole with paperwork attached. It also helps to keep the reason specific enough that a stranger can judge whether it still holds.
  • Your finding list is short but the team still ignores it. What do you check?
    Check who receives it and whether the entries name an action. A list addressed to everyone is owned by no one, and an entry that states a fact without a next step is read as commentary. Also check the dismissal history: if past entries were waved away, the list is carrying a reputation the current entries did not earn.

saying these in an interview costs you the question

  • Treats every unlinked case as a defect to be closed
  • Reports the whole product's findings on every run
  • Argues the same exception again at every review
  • Publishes one list to everybody and nobody owns it
  • Blames readers for ignoring a report that is mostly noise