Why do long test plan documents go stale faster than the code they describe?
answer
- Documents have no failing build
- Nothing signals that prose is wrong
- Copied facts decay on their own clocks
- Length equals surface area for drift
- Keep decisions, link the changing facts
basics
~10 sNothing forces a document to change when reality does. Wrong code fails a build; a wrong sentence fails nothing, so a long plan's duplicated facts drift silently until nobody trusts the document.
solid answer
~50 sThree forces drive the rot. **No execution feedback**: wrong code fails a build, a wrong sentence fails nothing, so errors accumulate without a signal. **Duplication**: long plans copy facts that live and change elsewhere - environment lists, defect counts, pipeline steps - and each copy decays independently of its source. **Incentives**: a document written to satisfy a gate is finished at the moment it is approved, not maintained afterwards. The counters are structural rather than moral: keep the document to the decisions only, link to live sources instead of copying them, date and name an owner for each section, delete sections nobody has cited, and review when the way of working changes rather than on a calendar tick. Above all, treat a stale document as a live risk - people act on it in onboarding and in reviews long after it stopped being true.
go deeper
Be ready to say why a document goes wrong more quietly than code does: nothing runs it, so nothing fails when it is untrue. One concrete example of a stale section is enough at this level.
Explain the duplication mechanism - copied facts decay independently of their source - and give the practical counter of linking to the live source instead of pasting a copy.
Show judgement from a real delivery: what you deleted from an inherited plan, what you replaced with links, how you handled the person who wanted the sections kept, and what the stale sections had already cost.
Own the incentive design. Say who is accountable for a document being true, what makes maintenance rational rather than heroic, and how you would prove to an auditor that thin documents plus live sources beat thick stale ones.
## The mechanism, not the morality The weak answer to this question blames the team for not updating the document. The strong answer treats rot as a predictable property of inert prose and designs around it. **Code has an oracle; a document does not.** When code contradicts reality, something fails - a build, a check, a request in production. When a sentence contradicts reality, nothing happens. The error stays, is read, is believed, and is propagated by the next person who copies the section into their own plan. Everything below follows from this asymmetry. **Duplication multiplies the exposure.** A long plan tends to restate facts that are already recorded and already changing somewhere else: which environments exist, which defects are open, which steps the pipeline runs, which cases the team holds. Every one of those copies has its own decay clock and no link back to its source. A one-page plan that links out has almost nothing to go stale; a thirty-page plan that copies everything has thirty pages of exposure. **Volume hides the decisions.** In a long inherited plan, the handful of sentences that actually decide something are buried among headings inherited from a template. Readers stop reading, so mistakes are not spotted, so nobody corrects them, so the document gets less trustworthy, so even fewer people read it. That loop is the rot. **Incentives finish the job.** If the document exists to pass a gate, its useful life ends at approval. Nobody is measured on whether it is still true in week six, and the release it describes has by then changed scope twice. ## What it costs A 4-person team inheriting an airline seat-map service found a 31-page plan whose risk section described a mitigation for a dependency that had been retired eleven months earlier, and whose environment section named two hosts that no longer existed. A new joiner spent most of a day chasing the retired dependency before someone senior said 'ignore that document'. That sentence is the real cost: once a team says it out loud, the organisation is paying to produce a document it has agreed not to believe - and the residual danger is that outsiders, auditors and joiners still believe it. A related trap: the plan named an intermittent timeout on roughly 2.7% of seat-map reads as an open risk, which had in fact been fixed two releases earlier. Stale risks are worse than missing ones, because attention is finite and gets spent on the phantom. ## How to fight it **Keep only decisions.** A plan should hold what was decided, by whom, and what was deliberately not done. Decisions age slowly and are worth writing. Facts age fast and should be referenced. **Link, do not copy.** Point at the live source for anything counted or generated. If a reader needs the current defect list, the plan should send them to where it lives, not attempt to hold a copy. **Date and own every section.** A section with a date and a name can be judged; an anonymous undated section cannot. When the owner leaves, the section is up for deletion by default. **Delete aggressively.** Track which sections anyone has actually cited during the delivery. Sections nobody consulted are the ones to cut at the next review - appending forever is what produced the 31 pages. **Review on change, not on calendar.** A yearly review of a document nobody reads produces a yearly rubber stamp. A review triggered when the way of working changes catches the thing that actually invalidated the text. **Watch for the tells.** The plan was not edited once during the release it governs. Two documents disagree and nobody noticed. Questions it supposedly answers are answered in chat instead. A section describes a system nobody currently operates. Any one of those means the document has stopped being maintained, whatever its approval status says. ## The audit objection Someone will point out that an external obligation requires a written plan, so it cannot simply be cut. That changes the form, not the principle. Auditors want evidence that decisions were made, by whom, and that they were followed; they are not helped by a copied environment table that is wrong. Keep the decisions and their owners in the document, keep the volatile facts behind links or generated appendices, and the document both satisfies the obligation and stays true. A signed document is evidence of a signature, never evidence that testing happened - a candidate who conflates those two has not been near a real audit.
- How do you decide whether a fact belongs inside the document or behind a link?Ask who owns the fact. If another system is its home and keeps changing it - the defect list, the environment inventory, the pipeline definition - link out; a copy will disagree with the source and the reader cannot tell which is right. Keep in the document only what the document itself decides: scope, deliberate omissions, the risks accepted and by whom. Decisions have no other home, so they must live there.
- An external audit obliges you to keep a written test plan. Does that change how you fight rot?It changes the form, not the content rule. Auditors want evidence that decisions were made, by whom, and that practice matched them. So keep decisions and owners in the document, and push volatile facts behind links or generated appendices that cannot drift from their source. A copied table that is wrong helps nobody, and a signature proves only that someone signed.
- What is the earliest signal that a plan has already rotted?Nobody edited it during the delivery it covers. A live document acquires edits - scope moved, a risk closed, an assumption broke. Zero edits across a whole release means it was written for approval, not for use. Close behind: people answer its questions from chat instead of citing it, and two documents contradict each other with no one noticing.
saying these in an interview costs you the question
- Blames the team rather than the document's design
- Answers with a longer or stricter template
- Copies pipeline configuration into the plan
- Only appends sections and never deletes any
- Treats an approved plan as evidence testing happened
- Reviews on a calendar tick regardless of change