A tester recorded a pass in an open cycle, and the next day the case's steps were rewritten in a repository whose cycles follow the master case live. What is that pass now evidence of, and how would you handle it?

## Why the pass is not simply wrong, and not simply fine A recorded outcome is a statement about three things at once: a **build**, a **case text**, and a moment in time. A live-reference cycle keeps the first and third stable and lets the second move. After the rewrite the repository renders the item as *current steps, verdict: pass*, and a reader has no way to see that those two halves came from different days. So the honest reading of the pass is narrow but real: **some earlier wording of this case passed against that build**. It is not worthless - the tester did execute something and did form a judgement. It simply cannot speak for the text now on screen, and the danger is that every downstream count treats it as if it can. Two failure modes follow from getting this wrong. Treating the pass as valid means shipping on evidence for steps nobody ran. Treating every edited case as void means re-running a cycle because somebody fixed a typo, which is expensive enough that teams stop doing it and quietly slide back to the first failure. ## Working out what actually changed The whole decision turns on the size of the edit, so establish that before anything else: 1. **If the execution pinned a revision**, render the case as of that revision and diff it against the current text. This is a lookup and takes seconds per item. 2. **If it did not, but the case keeps a change history**, use the entries between the result's timestamp and now to reconstruct what moved. 3. **If neither exists**, you cannot know. Default to re-executing, and treat the gap itself as the finding worth fixing. Then classify each change: | Kind of edit | Effect on the recorded pass | Action | |---|---|---| | Spelling, formatting, clarified prose | none - the same behaviour was checked | keep the result | | Reworded expected result, same meaning | none, but note it | keep, with a comment | | Changed expected result | verdict no longer applies | re-execute | | Step added, removed or reordered | part of the case was never run | re-execute | | Different test data or preconditions | a different scenario was run | re-execute | The unit of the decision is the **item**, not the cycle. A cycle where four cases were rewritten does not need re-running; four items do, and only those where the change was material. ## Handling the results you keep and the ones you drop - **Do not silently keep** a material-change pass. If the product allows it, add a comment or a marker recording that the item was executed against superseded text, so the next reader is not misled by the display. - **Do not silently delete** the ones you drop. Reset the item to untested for the new text, but leave the old execution in place - it is genuine evidence about the previous wording and about the build it ran on. - **Fix the totals, not just the item.** Cycle summaries, coverage views and release reports have already counted that pass. Whatever you decide has to reach the number a stakeholder will read, or the correction exists only in your head. - **Say what you did.** "Three items re-executed after their expected results changed; eleven kept after wording-only edits" is a defensible sentence. "The cycle is green" is not, once you know cases moved under it. ## Stopping it happening again The incident is a symptom of a combination, not of one careless edit: a repository that follows live edits, plus cycles that stay open long enough for editing to overlap execution. Attack whichever half is cheaper for the team: - **Shorten the overlap.** A cycle that opens and closes inside a day is very hard to edit out from under, whatever the model. - **Route the edit away from the case under execution.** Author the rewrite as a new definition and let the current cycle finish on the old one, rather than mutating the object people are executing. - **Agree an edit freeze** on the folders a cycle draws from while it is open. It costs nothing to implement and everything to enforce, so it works only where the editing group is small and the convention is genuinely shared. - **Ask for the pin.** If executions record the revision they ran against, this incident becomes a five-minute triage instead of an argument. - **Reconsider the model.** If drift keeps recurring, the repository's snapshot behaviour - not the team's discipline - is the thing to change. The interviewer is usually listening for one thing above all: that you separate *what the evidence supports* from *what the screen implies*, and that you spend re-execution effort in proportion to the edit rather than uniformly.