A tester records a passing result in a case management tool, but the requirement-coverage panel still shows the old percentage — what explains the lag, and how do you confirm it?

## Two ways a coverage figure gets produced Every coverage panel answers the same question, but products answer it with one of two very different machines, and which one you are on determines what a lag means. - **Recompute on read.** Opening the panel triggers the query: walk the filtered requirement population, follow each requirement's links, look at the executions in scope, apply the covered condition, produce the fraction. The number is correct as of the moment it rendered. The cost is a join across three growing sets, paid on every open, by every reader. - **Stored roll-up.** The fraction — or a per-requirement covered flag it is summed from — is persisted and served directly. Reads are cheap and constant-cost. Correctness now depends entirely on the refresh path: something must notice a change and update the stored value. Most products that survive at scale end up on the second design, or on a hybrid: recompute for a small population, stored roll-up above some size, with a manual rebuild available. That hybrid is worth knowing about because it produces the confusing symptom where a narrow filter shows the new result immediately and a wide one does not. ## What can break a stored roll-up The refresh path is where the staleness lives, and it fails in recognisable ways: 1. **The refresh is scheduled, not immediate.** The figure is correct as of the last run and will be right later. Nothing is broken; the panel is simply a periodic report and nobody said so. 2. **The refresh event was missed.** Result recording emitted a change that the roll-up never consumed — a dropped message, a failed job, a restart mid-batch. This one does not self-heal; the value stays wrong until something forces a rebuild. 3. **The write path bypassed the refresh.** Results imported in bulk, or written through a different route than interactive recording, sometimes skip the hook that ordinary recording fires. Bulk import is the classic case. 4. **The change was not on the watched path.** Adding a *link* also changes coverage, but some implementations refresh on result changes only, so a newly linked requirement stays uncovered in the panel until the next full rebuild. 5. **A cache in front of the roll-up.** The stored value updated; the reader is served an older rendering. ## Confirming it, rather than guessing The diagnosis is a bisection between three layers — the result, the link, and the roll-up — and it takes only a few minutes: - **Open the individual requirement** and read its links and its executions. This is the ground truth the panel summarises. If the passing run is visible here and the panel disagrees, the summary is stale and the underlying data is fine. - **If the requirement does not show the run either**, stop suspecting freshness. Either the result was recorded against a different case, or against a different cycle than the panel is scoped to, or the case is not actually linked to this requirement. That is an ingest or linking problem wearing a staleness costume. - **Re-open the panel with a much narrower filter.** On a hybrid implementation a small population may take the recompute path and show the true figure, which both confirms the diagnosis and gives you a correct number to work with now. - **Check whether other recent results also fail to appear.** One missing result suggests a single dropped refresh; every result since a given time suggests the refresh path stopped, which is an operational fault rather than a data one. - **Force a rebuild if the product offers one**, then re-read. If the figure corrects, the stored value was stale; if it does not, the data really does not support the number you expected. ## Living with a figure that is not computed on read Once you know a panel is roll-up backed, the working habits change: - **Do not read coverage as a live signal during an execution push.** It is a report with an unstated as-of time, and treating it as a live counter produces exactly the false alarms described above. - **Ask what the as-of time is** and, if the panel does not display one, treat the absence as a defect in the reporting rather than a reason to trust the number more. - **Rebuild before a release conversation**, so the figure you quote and the links behind it agree. - **Distrust upward jumps most.** A stale figure that is too low creates noise and gets investigated. A stale figure that is too high — because a link was removed or a case was deleted and the roll-up never noticed — creates false confidence and gets quoted. The general principle behind all of it: a summarised number and the records it summarises are two different artefacts with two different update paths, and any disagreement between them is information about the pipeline, not just an inconvenience.