What is an approval test, and how does it differ from a test with hand-written assertions?

### The problem it solves A conventional test states its expectation in code: you write the value you expect, run the code, and the framework compares the two. That works while the expectation is small enough to type and stable enough to be worth typing. It stops working when the output is a rendered receipt, a generated report, a forty-field serialised record, or the text a generator emits. Hand-writing those expectations is slow, the test becomes a wall of quoted text, and every intended change means editing the wall by hand. Approval testing inverts the order of the work. Instead of writing the expectation and then running the code, you run the code, look at what it produced, decide whether it is right, and **store that artefact as the expectation**. From then on the test's only claim is: today's output is identical to the artefact a human approved. ### The two artefacts Every approval test case owns a pair of files: - the **approved** artefact — committed next to the test, reviewed like source code, and the only expectation the test has; - the **received** artefact — written on every run, transient, and excluded from version control. The run loop is small: produce output, write it to received, compare received with approved. On a match the received file is discarded and the test passes. On a mismatch the test fails and *leaves the received file on disk*, usually launching a diff so the difference is visible immediately rather than as a truncated string comparison in a log. ### What the assertion actually says This is the point candidates most often miss. A green approval test says **"unchanged since approval"**. It does not say "correct". Two consequences follow. First, an approval test can never be stronger than the review that created it. If the reviewer approved a receipt with a wrong tax line, the suite now defends the wrong tax line and will go red the day someone fixes it. Second, it detects *any* change, including the ones you wanted. An intended behaviour change turns the test red by design; you re-run, read the diff, confirm it is the change you meant, and re-approve. That re-approval step is the discipline. Approving without reading converts the whole technique into an expensive way of recording whatever the code currently does. ### Where it earns its place Approval tests are strongest on output that is wide, structured and human-judgeable but not concisely expressible: - rendered documents — receipts, invoices, statements, printable summaries; - generated text — emitted source, configuration, templated messages; - serialised records with many fields, where a hand-written expectation would repeat the whole structure; - error and message catalogues, where the exact wording is the behaviour; - combinatorial output, where one artefact per input combination covers a wide surface for the cost of one review each. They are also the cheapest wide net you can throw over behaviour you are about to change: lock the current output, change the code, and every unintended difference shows up as a diff. ### A worked shape Consider the receipt renderer behind an online bookstore checkout. A **340-case regression pack** holds one order shape per case — single item, mixed physical and downloadable items, gift wrapping, a partially refunded order, a multi-currency edge. Hand-asserting each rendered receipt would be thousands of lines of quoted text. As approval tests, each case is a few lines that render the order and compare against its approved receipt. When a loyalty-discount line is added, 118 of the 340 artefacts change, and the review is a read of 118 diffs rather than an edit of 118 expectations. ### What it costs Three costs are real and are what interviewers probe. **Determinism** — anything that varies between runs (a clock reading, a generated identifier, an unspecified collection order) must be normalised or injected, or the test fails on unchanged code. **Granularity** — approving too much makes diffs unreviewable and couples many artefacts to one change; approving too little rebuilds hand-written assertions with less intent. **Review decay** — a team that re-approves reflexively still has green tests and no oracle at all. ### Relationship to assertion-based tests They complement rather than replace each other. A rule such as "the total equals the sum of the line amounts minus the discount" belongs in a written assertion: it states intent, survives an unrelated whitespace change, and explains itself when it fails. An approval test on the same receipt would go red for a changed column width and would never tell you *why* the output should look that way. Sensible suites carry both: written assertions for the invariants that matter, approval tests for the shape and detail nobody wants to type.