Your case repository's open cycles follow live edits, and testers keep asking whether a recorded pass still refers to the steps they ran. How would you decide whether to move to snapshot-at-cycle-creation?
answer
- which failure can we absorb
- measure the overlap, not the complaint
- provenance is cheaper than a model change
- propagation arrives with the snapshots
- history cannot be retro-frozen
basics
~20 sMeasure the overlap first: how often cases are edited while a cycle using them is open, and how long cycles stay open. If the overlap is rare, buy provenance instead - record the revision each result ran against. Switch models only when drift is routine and costly.
solid answer
~50 sTreat it as a trade, not an upgrade. Snapshots buy a stable text per cycle and cost you propagation: a real correction never reaches cycles that already copied the mistake, and storage grows per cycle. Before paying that, size the problem - how many cases are edited during an open cycle, how long cycles stay open, and how often a result was actually questioned. Then consider the cheaper interventions in order: record the revision on each execution, which answers the testers' question directly; shorten cycles so editing and execution barely overlap; route rewrites into a new definition rather than the one under execution. Switch the model when drift is routine rather than occasional, or when the people who read the results are outside the team and cannot ask. Plan for the fact that already-open cycles cannot be retro-snapshotted: a copy taken today captures today's text, not what an old result was formed against.
go deeper
Recognise that this is a choice a repository makes, and that neither answer is simply correct - each removes one problem by accepting a different one.
Explain the mechanics you would rely on: recording the revision each result ran against, and how a snapshot changes the path a corrected case takes to reach an open cycle.
Show that you would measure edit-during-open-cycle rate, cycle duration and materiality before proposing a change, and that you would try provenance and shorter cycles first.
Own the transition and its scope: mixed cohorts, a stated propagation policy, results that stay ambiguous forever, and whether the model is set per project rather than globally.
## Frame it as a trade, not a fix The two models fail in opposite directions, and moving to snapshots does not remove a problem - it exchanges one for another. | | Live reference (today) | Snapshot at cycle creation | |---|---|---| | A result's wording | can move after the fact | stable for the cycle's life | | A genuine correction | reaches everyone at once | stranded outside open cycles | | Testers mid-cycle | can be handed different texts | all read the same text | | Storage and search | one definition | grows with cycles times cases | | The failure you get | verdicts under text nobody ran | stale steps executed for weeks | So the real question is not "which is better" but **which failure this organisation can absorb**, given how it works today. ## Measure before you choose The complaint is a signal, not a measurement. Three numbers turn it into a decision, and the repository's own history usually holds all three: 1. **Edit-during-open-cycle rate.** How many cases were edited while a cycle containing them was open, over the last few releases? This is the only number that says whether drift is real here rather than theoretically possible. 2. **Cycle duration.** Drift needs a window. A repository where cycles open and close within a day has almost no exposure whatever its model; one where cycles stay open for a whole release has a lot. 3. **Materiality of those edits.** Sample them. If nearly all were wording and formatting, the complaint is about *confidence*, not about wrong results, and confidence is fixable far more cheaply than the model is. Add one qualitative input: **who reads the results, and how long after the fact.** A team reading its own cycle this week can ask the tester what they saw. Someone outside the team reading it next quarter cannot, and their inability to ask is what makes provenance structural rather than a convenience. ## Try the cheaper interventions first In rough order of cost, each of these addresses part of what the testers are actually complaining about: - **Record the revision on each execution.** If the product supports it, this answers the literal question - *which steps did I run?* - without changing how anything is stored or how corrections propagate. It is nearly always the first thing to reach for. - **Shorten the cycles.** Smaller, faster cycles reduce the overlap window directly and pay off in several unrelated ways. This is the intervention with the best side effects. - **Route the rewrite elsewhere.** Author a substantial change as a new definition and let the running cycle finish against the old one. This costs authoring discipline rather than storage, and it keeps corrections flowing normally everywhere else. - **Freeze edits on the folders a cycle draws from** while it is open. Cheap to state, hard to enforce, and reliable only where the editing group is small and shares the convention. - **Change the model.** The largest, least reversible option, and the one that also changes how every future correction behaves. ## What switching actually costs If you do switch, go in knowing four things: - **Open cycles cannot be retro-snapshotted.** A copy taken on the day of the change captures the text *as of that day*. For results already recorded, the wording they were formed against is simply gone unless a revision was recorded at the time. Expect a cohort of results that stays ambiguous forever, and say so rather than implying the switch fixed history. - **You inherit the propagation problem the same day.** Somebody will find a wrong expected result in week one and discover the fix reaches no open cycle. Have the answer ready - a refresh path, or a policy that material corrections mean re-adding the case - before the first person hits it. - **Mixed cohorts are unavoidable during the transition.** Cycles opened before and after the change behave differently, and readers will not know which is which unless the cycles say so. - **The line may be partial.** Many repositories copy only the steps and leave titles, folders, links and attachments live. Confirm empirically which fields the copy covers before promising anyone the cycle is frozen, because the promise is only as good as the least frozen field. ## Scope the decision honestly It is rarely one answer for an entire organisation. A repository that serves both a fast-moving product team and a long-lived certification effort may reasonably want different behaviour per project, and most products allow that granularity even when the discussion is framed globally. Decide per repository or per project, write down which model each one uses and why, and revisit it when cycle duration or editing volume changes - the two inputs that actually drive the answer.
- If you switch to snapshots, how should a genuine mistake in a case reach cycles that already copied it?Decide it in advance rather than per incident. The workable options are a per-item refresh that resets any result recorded against the older copy, or a policy that a material correction means removing the item and re-adding the case. Both are defensible; what is not is fixing the master, telling people it is fixed, and leaving open cycles executing the mistake.
- Would you apply the same model to every project in one repository?Not necessarily. A team running short cycles on a fast-moving product loses little to live references and gains from corrections propagating instantly; a long-lived effort whose results are read by outsiders months later needs a stable text per cycle. Most products let the behaviour be set per project, so decide per project and record which model each one uses.
saying these in an interview costs you the question
- Treating snapshots as strictly better than live references
- Switching on one complaint without measuring drift
- Believing the switch repairs results already recorded
- Ignoring that corrections stop propagating after the change
- Choosing globally when the products allow per-project behaviour