Test Data & Environments

An in-memory database stand-in starts in milliseconds, needs no external service and resets cleanly between cases, so a suite runs fast anywhere. It hides every behaviour where the production engine differs: dialect, type coercion, collation, constraint enforcement and locking.

Hand-built fixtures state only the few records a case needs, so they read clearly and carry no privacy risk. Generated records buy volume and variety cheaply. Production-derived extracts show real shapes and skew, but must be cut down and masked first.

A failing assertion aborts the test body, so cleanup written after it never executes and the records it created leak into later tests. Register teardown as an after-each hook or a resource-scoped block so it runs on both exits.

Random data makes a failure depend on values nobody chose. Recording the seed and printing it in the failure output lets anyone re-run the identical values, so the failure can be reproduced and fixed instead of guessed at.

Service virtualization replaces a dependency you do not control with a programmable stand-in that answers over the same protocol and address. Reach for it when the real service is unavailable, costly, rate-limited, or cannot be pushed into the state a test needs.

Pre-production usually differs in data volume and shape, node count, hostnames and certificate chains, proxy hops, vendor sandbox accounts, permissions, and configuration. Each difference is a class of defect the pre-release run cannot see.

A child row can only point at a parent that already exists and whose identifier is known. Generating in dependency order - parents first, then the rows referencing them - makes every reference resolve to a real row.

Substitution swaps in a plausible replacement and keeps type and shape; shuffling keeps the column's real distribution but detaches values from their rows; banding keeps magnitude and order while losing precision; blanking keeps nothing and forces the empty-value path.

Masking covers the stored dataset, not what a test run writes out. Real values still arrive from unmasked upstream systems, skipped fields and hand-typed input, then land in log lines, screen images, recorded traffic and failed-check comparison output.

Test flows join records across systems on that identifier. If each system replaces it differently the join finds nothing, cross-system journeys stop at the second hop and reconciliation disagrees - failures manufactured by the masking, not by the code.

Direct identifiers are only part of identity. Attributes that look harmless alone -- birth date, postcode, job title, the month an account opened -- combine into a pattern very few people share, and a unique combination points at one person.

Timezone, locale and default encoding are process-wide inputs that date conversion, formatting, parsing and byte-to-text conversion read implicitly. Left unpinned, the same code yields different results on a different machine, so a suite passes locally and fails on someone else's.

Configure the stand-in to answer badly: a delay just above the client's configured timeout, an error or throttling status, a dropped connection, or a malformed body. Then assert on the client's retries, error mapping and stored state.

Realistic generation reproduces the common middle, not the rare tails. Absent optional fields, exact boundary amounts, right-to-left names and rows flagged deleted appear far too rarely to arrive by chance, so each must be planted deliberately rather than waited for.

Authored rules guarantee exactly the constraints someone wrote down, and nothing else. A generator fitted to real records reproduces that data's shape, including correlations nobody stated, but guarantees no particular constraint holds in every produced row.

Whether any legitimate workflow must recover the original value. If nothing needs the real person back, use an irreversible pseudonym; a reversible token is only worth the mapping store it creates when a named workflow genuinely needs the link.

A whole-database copy drags every record's privacy obligation into a weaker environment, grows until the restore is the bottleneck, and gives an unstable target: rows a case asserts on change between refreshes. A masked subset costs far less.

A label ties a failing test to the exact planted row it hit, turning a bare count mismatch into a named situation. Because the dataset is rebuilt, the label must come from the build definition, not from a store-assigned identifier.

Hand every test a fresh builder, compute time-dependent and unique defaults at build time rather than once at load, deep-copy nested objects, and derive variants by copying the builder instead of mutating a shared one.

The suite tests the code model's schema while production runs the migration history. Nothing executes the migrations, so a bad backfill, a wrong ordering or an unsafe alteration reaches production first, and the generated schema silently drifts.

Redaction destroys a value: the field becomes a blank or a constant, and nothing can be inferred from it. Pseudonymisation swaps each value for a stable surrogate, so equal values stay equal and joins and grouping still work.

Roll back when everything the test touches happens inside one transaction on one connection, because that reset is nearly free. Truncate and reseed when the code under test commits, spans connections, or when rollback would hide commit-time behaviour.

Replace it with a value that satisfies the same rules - same length and character classes, same internal segments, a recomputed check digit - then run the replacement through the same validation routine the product uses before publishing the dataset.

A later sweep runs after the value is already durable, already replicated and possibly already read. Filtering in the write path means the value is never stored at all, and that filter can be proved by a check rather than hoped for.

The substitute is computed, not assigned: normalise the real value, run it through a keyed one-way transformation with one shared secret, then shape it to the field. Same input and secret, same result - no state shared between jobs.

A column pass replaces values in fields somebody classified. It cannot read inside free-text notes or stored documents, and it does not recompute columns built from the values it changed, so names, numbers and reconstructable values survive in all three places.

It evaluates matching predicates over the incoming request - method, path, query, headers, body - and serves the matching rule's canned response. An unmatched request must fail loudly with a distinctive status and a logged diagnosis, never a blanket success.

Values that name the environment — endpoints, credentials, account identifiers, resource sizes — are meant to differ. The key set, defaults, precision settings, timeouts and feature-flag states are not. Divergence in behaviour-changing settings is what escapes to release day.

Anything that filters, sorts or groups by time. Every range query returns all rows or none, ageing paths never fire, ordering ties are decided by the store, and the causal orderings the product depends on are neither exercised nor violated.