In a requirement-coverage panel in a test-case management product, how does the reported figure differ when covered means a link exists, versus executed in the selected cycle, versus passed?
answer
- three nested subsets, one population
- linked, executed, passed descend
- the cycle scope resets executed
- the gaps localise the problem
basics
~20 sEach condition is stricter than the last, so over one population the three readings form a descending ladder: a link counts intent, executed counts work actually done in the chosen cycle, passed counts behaviour confirmed. The gaps between them are the diagnosis.
solid answer
~50 sThe three conditions are nested subsets over the same requirement population, so linked is always at least executed, and executed is always at least passed. A link existing says only that somebody agreed a case addresses this requirement — it can be satisfied by a case nobody has ever run. Executed in the selected cycle adds a time scope: the linked case must have a result recorded in *this* cycle, so last cycle's runs stop counting and the figure resets when a new cycle opens. Passed adds an outcome scope on top: a failed, blocked or skipped run keeps the requirement uncovered. Reading all three is more useful than reading any one, because the drop from linked to executed measures planning that has not been carried out, and the drop from executed to passed measures behaviour that is not working.
go deeper
Know that covered is a setting, not a fixed meaning, and that a link existing is far weaker evidence than a passing run in the current cycle. Say which of the three a figure used before repeating it.
Be ready to explain the nesting — linked is at least executed is at least passed — and why a cycle-scoped condition makes coverage reset rather than accumulate across cycles.
Demonstrate diagnosis from the gaps: linked-minus-executed points at scheduling, executed-minus-passed points at behaviour or environment, and the enumerated uncovered rows are what you actually work from.
Own which reading is published and how, knowing that a linked-scoped target is trivially satisfied by administrative link creation and a passed-scoped target quietly rewards shallow cases.
## Three conditions, one population, three numbers A requirement-coverage panel in a case-management product lets you choose what makes a requirement count as **covered**. The population — the requirement set chosen by the saved filter — usually stays the same while you switch conditions, so all three readings share a denominator and differ only in the numerator. The conditions form a strict nesting: 1. **A link exists.** Some test case is associated with the requirement. Nothing is asserted about whether it has ever been run, or by whom, or when. 2. **Executed in the selected cycle.** A linked case has a result recorded within the cycle the panel is scoped to. The result may be any outcome. 3. **Passed.** A linked case has a recorded result in that cycle whose outcome is a pass. Because each condition implies the one above it, the numbers can only descend: `linked >= executed >= passed`. A panel that ever shows executed above linked, or passed above executed, is telling you the three are being computed over different populations or different cycles — which is a real and common configuration accident, not a paradox. ## What each reading actually measures | Condition | The question it answers | Its characteristic failure | |---|---|---| | A link exists | Has anyone planned to test this? | Satisfied by a case that has never run, or is permanently skipped | | Executed in the cycle | Did we do the work this time round? | Satisfied by a failing or blocked run — work happened, behaviour did not | | Passed | Did the behaviour we agreed actually work? | Silent about depth; one shallow passing case covers the requirement | The **cycle scope** on the second and third conditions is the part that surprises people most. It means coverage is not a monotonic, accumulating property. Open a fresh cycle and an executed-scoped or passed-scoped panel drops toward zero, because no case has run *in that cycle* yet, even though nothing about the links or the historical results changed. A linked-scoped panel does not move at all. Anyone reading the figure across a cycle boundary without knowing which condition is configured will conclude that coverage collapsed overnight. ## The gaps are the interesting signal The single reading everybody quotes is the least informative use of the panel. Read all three and the differences localise the problem: - **Large gap between linked and executed** — the plan exists but was not carried out in this cycle. Causes are usually scheduling: the cases were never pulled into the cycle, the cycle is still in flight, or the linked cases sit in a suite nobody assigned. - **Large gap between executed and passed** — the work was done and the behaviour is not right, or is not runnable. Failures, blocked cases and environment problems all live in this gap, and they look identical in it, which is why the uncovered list matters more than the delta. - **Linked near-complete and both others near zero** — a strong sign the links were created as a bulk administrative exercise rather than as a by-product of building the cases. - **All three equal and high** — either genuinely good, or the population has been filtered down to the part of the product that was already finished. ## Why the choice of condition is a governance decision, not a preference The condition you publish sets the incentive. Publish linked coverage and the cheapest way to move the number is to create links, which takes minutes and improves nothing. Publish passed coverage and the cheapest way to move the number is to link shallow cases that pass easily. Neither is dishonest by itself, but both are what the measure rewards, and a team under pressure will find them. The mitigations are ordinary: - Quote the condition **with** the number, every time, so a reading is reconstructible. - Publish more than one reading, so the gap stays visible and cannot be closed by definition-shopping. - Treat the enumerated uncovered requirements as the deliverable, and the percentage as the index into it. ## A detail that bites in practice What counts as an execution and what counts as a pass is not always as literal as it looks. Some repositories treat a requirement as executed if **any** linked case ran, others only if **all** linked cases ran; the same choice exists for passed. Those two policies produce very different figures for a requirement with several linked cases where one fails. There is no universal default across products, and it is rarely displayed next to the percentage, so it is worth establishing once for whichever repository you are reading rather than assuming. The same applies to non-terminal outcomes: a retest-pending or in-progress result may count as executed, or may not, and blocked is sometimes grouped with not-run rather than with executed. When two panels over the same data disagree, this is one of the first places to look.
- Why does a passed-scoped coverage figure drop sharply the morning a new cycle opens?Because the condition is scoped to the selected cycle, and no case has recorded a result in a cycle that has just been created. The links are untouched and last cycle's results still exist, but they no longer qualify. A linked-scoped panel over the same data would not move at all, which is why the condition must travel with any figure quoted across a cycle boundary.
- A requirement has four linked cases; three pass and one fails. Is it covered?It depends on a policy the panel rarely displays: some repositories mark the requirement covered if any linked case passed, others only if all of them did. The any-policy makes coverage easy to reach and lets a real failure hide inside a covered requirement; the all-policy is stricter but punishes a requirement whose broad linked case is unrelated to the change. Establish which one your repository applies before quoting the figure.
saying these in an interview costs you the question
- Assumes coverage always means a passing run
- Thinks a link existing implies the case ever ran
- Misses that executed and passed are cycle-scoped
- Quotes one reading and ignores the gaps