skip to content

What makes a scenario report generated from the last run 'living documentation'?

level: juniorimportance: must knowfreq 62%

answer

  1. Where does the published text come from?
  2. Nobody edits it by hand
  3. Every run regenerates it
  4. Behaviour drifts, a page turns red
  5. Green means these scenarios, not everything

basics

~20 s

The document is generated from scenarios that actually executed, so it describes only behaviour the suite exercised. When the system changes and the scenario does not, the scenario fails and the page visibly breaks instead of quietly going stale.

solid answer

~40 s

Living documentation has no hand-maintained source: the published text is generated from business-readable scenarios plus the results of the run that just executed them. That mechanism gives it the property people actually want — it cannot disagree with the system in silence. If behaviour changes and nobody updates the scenario, the scenario fails and the corresponding page renders red on the same build, so staleness is announced rather than discovered months later. Publishing it usefully means stamping provenance on the artefact: the revision, the run time and the environment it ran against. The honest limits are worth stating too. A fully green report documents only the behaviour that has scenarios, says nothing about behaviour nobody wrote one for, and guarantees accuracy rather than readability.

go deeper

for a junior

Be ready to state the mechanism in one sentence: the document is generated from scenarios that ran, so it cannot be wrong without a scenario failing. Contrast it with a hand-written page that goes stale with no signal.

for a middle

An interviewer expects you to explain how the artefact is produced — scenario text merged with per-scenario run results, stamped with the revision and environment — and to name what a green report does not prove.

for a senior

Show judgement about trust: which run the artefact is generated from, how not-executed scenarios are rendered, and why publishing an unstamped or partially-run report is worse than publishing nothing.

for a principal

Own the argument for whether the organisation should carry the wording discipline at all: generation is cheap, keeping hundreds of scenarios readable for outsiders forever is not, and the payoff depends on an audience that exists.

### What the phrase actually claims **Living documentation** is a description of a system that is *generated from artefacts that execute*, rather than written and maintained by hand. In a behaviour-driven setting those artefacts are the scenarios: business-readable descriptions of behaviour, each attached to automation that drives the real system. A publishing step takes the last run of that suite and renders it as a document — one page per feature, each scenario shown with its steps and the status the run gave it. The word *living* is doing precise work. It does not mean "frequently updated". It means the document has no independent existence: there is no source file a person edits to change the published text, and no version of the text that can be right while the system is wrong. If someone changes the behaviour and does not change the scenario, the scenario fails, and the corresponding page in the published document renders red. The documentation does not quietly drift out of date — it *breaks loudly*, in a place people are already looking, on the same build that introduced the change. ### The mechanism, step by step 1. Scenarios live in the same repository as the code, in business-readable text. 2. A run executes them against a known revision of the system. 3. The run emits a machine-readable result per scenario: passed, failed, or not executed. 4. A generator merges the scenario text with those results and publishes an artefact — usually a static site — stamped with the revision, the timestamp and the environment it ran against. Every property people want from living documentation comes from step 3 being real. If the report is generated from scenario text alone, with no run behind it, it is a prettier wiki page: it can be as wrong as any hand-written page, and nothing will say so. ### A worked example A four-person team owns a parcel-tracking gateway. Their published specification says: *Given a parcel has been scanned at a depot, When a customer requests its status, Then the response shows the depot's city and the scan time.* Someone adds a cache in front of the status lookup, and under a particular expiry path the gateway serves a stale-cache read: the previous depot, minutes after a newer scan. The scenario fails on the next run. The published page for parcel status now shows a red scenario with the failing step highlighted — so the document is announcing, to anyone who opens it, that the system no longer does what it says. A hand-written page describing the same behaviour would have stayed confidently, silently wrong for as long as nobody re-read it against the code. ### What it does *not* give you Three honest limits are worth saying out loud in an interview. **Completeness.** A green report documents the behaviour that has scenarios. It says nothing about behaviour nobody wrote a scenario for. Reading a fully green specification report and concluding "the system is fully specified" is the single most common misreading of the artefact. **Readability.** Generation guarantees the document is *accurate about the scenarios that ran*. It guarantees nothing about whether the prose is comprehensible to a reader outside the team. That depends entirely on how the scenarios were written, and it degrades quietly as scenarios accumulate incidental detail. **Audience.** A generated artefact that nobody outside the team ever opens is a build cost, not a documentation strategy. The generation is cheap; the discipline of keeping hundreds of scenarios worded for an outside reader is not. ### Why interviewers ask it It is the clearest articulation of what behaviour-driven work buys that ordinary automated tests do not. Automated checks and living documentation are the *same artefacts read by different audiences*: the team reads them as a regression signal, everyone else reads them as the specification of record. A candidate who can state that duality — and then name what it does not cover — is showing they understand why the business-readable layer is worth its translation cost, instead of reciting a slogan about executable specifications. ### How to answer well Lead with the mechanism ("generated from a run, so it cannot disagree with the system without failing"), contrast it with the failure mode of hand-maintained documentation (silent staleness), and close with the limit (green means *these* scenarios pass, not that the system is fully described). If you have published such a report in practice, mention what you stamped on it — the revision and the run time — because provenance is what turns a rendered page into evidence.

  • If a team publishes only the pages from scenarios that passed, what does the reader lose?
    The gaps. Filtering to passing scenarios turns the report into an advertisement: failing behaviour disappears rather than showing red, so the document silently regains exactly the staleness the mechanism was supposed to remove. Publish every scenario with its status, including the ones that failed and the ones that never ran.
  • What can a fully green generated specification report never tell a reader?
    Whether anything is missing. It describes the behaviour that has scenarios and nothing about behaviour nobody specified, so completeness cannot be inferred from it. It also says nothing about how comprehensible the prose is to an outside reader, or about qualities like performance and resilience that the scenarios do not assert.
  • Why stamp the revision and run time on the published artefact?
    Without provenance the document is undated evidence. A reader opening it next month cannot tell whether it describes today's system, and an auditor cannot tie a claim to a build. The header should carry the revision, the finish time, the environment and any selection the run applied.

A hand-written specification is a photograph of the system on the day someone took it. Living documentation is a live camera feed: when the scene changes, the picture changes, and if the feed drops you can see that it dropped.

saying these in an interview costs you the question

  • Calls it a wiki page that the team remembers to update
  • Says a green report proves the system is fully specified
  • Thinks scenarios written after release still produce living documentation
  • Assumes generation alone makes the prose readable to outsiders
  • Publishes the report with no revision, time or environment stamp

context