Why does uniformly generated seed data understate how a system behaves at volume?

## Volume without shape is only half the variable When a team decides to test at scale, the first instinct is a loop that inserts rows until the count is right. That gets the size honestly and the *shape* completely wrong, and shape is where most volume-sensitive defects live. A dataset has at least six shape axes worth reasoning about: 1. **Key skew (cardinality and hot keys).** How rows distribute across the values you group and filter by. Real distributions are lopsided; generated ones are flat. 2. **Collection-length distribution.** How many children an entity has. Typically a long tail: most entities tiny, a handful enormous. 3. **Value-length distribution.** Text fields that are short on average and occasionally very long. 4. **Absent and default values.** Real columns have gaps, defaults and legacy placeholders; generators fill everything. 5. **Variant mix.** A minority of records carrying an older or region-specific representation that the code handles on a separate path. 6. **Temporal distribution.** When rows were created — a decade of history versus everything inserted in the last hour, which changes clustering and how much of the data is actually warm. ## What each one hides when it is flat **Flat key skew** makes every grouped operation cost the same. If the truth is that one key owns a large share of the rows, then the request that touches that key does far more work than the average, and that request is the one your users complain about. A uniform seed has no such request, so the measured distribution of latencies is narrow and comfortable and completely unlike production, where the tail is driven by the hot key, not by the network. **A flat collection-length distribution** removes the worst-case entity. If your generator gives every account exactly 40 items, no code path ever handles the account with 90,000. That account is where the unbounded response, the quadratic rendering loop and the memory spike live. **A flat variant mix** removes the slow path entirely. This one is subtle and it is worth an example. A parcel-tracking gateway seeded 12.4 million parcels with a single canonical weight-and-date representation, and its volume run looked clean. In production, roughly 3.2% of inbound records arrived with a locale-dependent numeric and date format, and the gateway parsed those through a fallback that constructed a formatter per record. At development size the fallback ran a few hundred times a day and cost nothing measurable. At production volume it ran on hundreds of thousands of records per hour during ingest, and the ingest job — previously 12 minutes — began missing its schedule. The defect was not the parser; it was the seed data, which had no variants in it at all, so the path was never sampled. **Flat temporal distribution** distorts warmth. If everything was written moments ago, an unrealistic share of it is in cache and the run measures memory-speed access where production would go to storage. ## Building a profile instead of a count The practical method is to describe the shape as a small set of numbers and generate against it: - rows per key at the 50th, 95th and 99.9th percentile, plus the single largest; - the length distribution of the child collections, again by percentile; - the fraction of records in each format or status variant; - absent-value rates per field; - the age spread of records. Those are aggregates, so they can be shared without exposing anything sensitive, and they change slowly enough that a quarterly refresh is usually sufficient. (Where the underlying records come from, and how they are masked or subsetted, is a separate concern with its own handling; what matters here is the shape you are aiming at.) Then make the extremes explicit rather than hoping the random draw produces them. A good generator plants **named worst-case fixtures** — the account with the largest collection, the key holding the biggest share, the record in the rare format — so a case can assert against them by name and fail loudly when a change makes the worst case worse. Randomly hoping to hit the tail is how tail defects escape. ## What to assert once the shape is right Assert against the extreme, not the median. "The tracking history renders in under a second for the parcel with the longest history" is a useful oracle; "average render time is 90 ms" is not, because the average is dominated by tiny entities that were never at risk. Equally, report which entity produced the worst measurement, so the number is traceable to a fixture someone can reproduce. A small team can afford this. An 11-person team does not need a data-engineering function to keep five percentile figures and a variant mix in a checked-in profile file and regenerate a shaped dataset on a schedule; what it cannot afford is discovering the shape for the first time from a production incident.