Why is restoring a full production copy usually a poor source of test data?

"Just restore production" is the most natural-sounding test-data plan there is, and it degrades in a predictable order. Being able to walk an interviewer through that order — and through what replaces it — is the senior part of this topic. ### What goes wrong, in the order it usually bites **1. Privacy scope.** A full copy imports the complete personal-data estate into an environment with more accounts, more copies and fewer controls. Every field with a person behind it must be transformed correctly on every rebuild, forever, and the failure mode is silent: an extract that is *mostly* transformed looks exactly like one that is fully transformed. **2. Volume you do not use.** A school timetable planner run by an 11-person team holds 4,213 students, 187 teachers, 1,046 courses and 61,214 timetable slots — a 6.8 GB dump that takes about 23 minutes to restore. The suite's assertions touch perhaps a few hundred rows. Every one of those 23 minutes is paid on every refresh, every environment and every rebuild after a corrupting run, and the predictable consequence is that the team stops refreshing. Data that is nine months stale is no longer 'realistic', it is merely large. **3. The oracle stops being stable.** An **oracle** is whatever decides the observed behaviour is right. If a case asserts that a particular student appears in a particular timetable, the oracle now depends on a record a real user owns. Someone withdraws that student in production; the next refresh, the case fails, and an engineer spends a morning on a defect that does not exist. Suites built on a full copy accumulate these until people stop believing red results. **4. The cases you need are absent.** Production holds what happened. The boundary case — the enrolment that expires today, the teacher with no assigned room, the course at exactly its capacity limit — is often not present, or is present once and disappears next month. You end up constructing those records anyway, which means the full copy bought volume and cost you a stable target. **5. It hides nothing about correctness.** A full copy makes a suite feel thorough while the assertions still only cover the paths someone wrote a case for. Volume is not coverage. ### What replaces it The standard answer is a pipeline, not an artefact: **Subset.** Choose a root entity and take a slice around it, following relationships outward: one school, its 187 teachers, its enrolled students, the courses those enrolments point at, plus every reference and lookup table copied whole because they are small and everything depends on them. The output must be **referentially consistent** — no row pointing at a parent the subset omitted. The classic failures are dangling references from rows the traversal reached but whose parents it did not, cyclic relationships that need a two-pass insert, and rows whose validity depends on a sibling table nobody thought to include. **Transform.** Personal fields removed or replaced by consistent surrogates on the way out, never after landing. **Verify.** Checks against the produced subset before anything is allowed to use it: every foreign key resolves, no field still matches the shape of a real contact value, per-entity aggregate counts match expectations. This matters because these pipelines fail partway. On one such run, the transform rewrote the student table, then failed on the guardian table; the step rolled back only its own partial work and exited, leaving 3,912 guardian rows with real contact values and references that no longer matched the rewritten student identifiers. The exit status said 'failed'; the extract was loaded anyway, because nothing was gating on the data itself. **Pin and rebuild.** The subset is a build output, reproducible from the pipeline, versioned and disposable — never a hand-tended artefact that accumulates manual fixes nobody can reproduce. **Keep assertions off it.** The subset is background, giving realistic shapes, volume and neighbours. Each case still creates the specific records it asserts on. That single discipline removes the unstable-oracle problem entirely, and it is what lets the subset be resized or regenerated without touching a test. ### When a full copy is genuinely right Be fair to it, because interviewers listen for nuance. Rehearsing a migration on true volume; reproducing a defect that only appears at production scale or with production skew; capacity work where the whole distribution matters. Those are **short-lived, purpose-built, tightly controlled** copies with an owner and an expiry — not the standing source a regression suite runs against every day. The distinction is between a copy used as an experiment and a copy used as a foundation.