Why does a naive random generator explore almost none of a structured input space?

### The arithmetic of blind sampling A property is only as good as the inputs it sees. The common failure is not a wrong property but a generator that produces ten thousand inputs from one uninteresting corner of the space. Take the playlist service's batch endpoint, which at peak absorbs a 1,200-request-per-minute stream of playlist updates. A first generator builds a batch of update requests, each with a playlist id, a version number and a track list, every field drawn independently at random. It runs 12,800 batches and stays green. Then production shows corrupted playlists. The reason is visible the moment anyone looks at what was generated. Playlist ids were drawn uniformly from a huge identifier space, so **two requests in the same batch almost never touched the same playlist** - across 12,800 batches there were three collisions. The concurrent-update path, where two updates to one playlist must be applied in version order, was therefore exercised three times, and never with equal version numbers. The defect was an **ordering assumption**: the batch handler applied updates in whatever order the batch happened to arrive in and assumed that order matched version order. Real traffic at 1,200 requests per minute breaks that assumption constantly; a uniform generator practically never does. ### The four ways naive generation goes wrong **Validation death.** Independently random fields rarely satisfy structural constraints - a date inside a range, a checksum, a well-formed identifier, a total that matches its line items. If the property discards invalid inputs after generating them, most of the run is spent generating and throwing away, the effective sample is a fraction of what the count suggests, and the surviving inputs are the ones that were easy to hit by chance. **Lost relationships.** Almost every interesting behaviour depends on two parts of the input relating to each other: the same id twice, a timestamp equal to another, a range that overlaps another, a reference that resolves. Independent draws destroy all of them. **Degenerate size.** Unless size is deliberately grown, generated collections cluster small. Off-by-one behaviour at a cap of 43, pagination past the first page, and recursion deeper than one level are simply never visited. **Missing the boundaries.** The values most likely to break code - empty, zero, one, the maximum, a duplicate, the same value twice, an unusual character class - have negligible probability under uniform sampling of a large domain, yet they are where the defects are. ### What good generation does instead **Construct, do not filter.** Build inputs that are valid by construction: pick a track list first, then derive a consistent count; generate a version sequence rather than independent integers. Filtering is acceptable for a rare, cheap final check; it is a poor way to satisfy the main structure. **Shrink the pool to force collisions.** Draw playlist ids from a pool of eleven, not from the whole identifier space. This single change turns a collision from a once-in-4,000-batches accident into the common case, and it is usually the highest-value edit anyone makes to a generator. **Inject the boundaries deliberately.** Mix in the empty collection, the single element, the value at the cap and just past it, the repeated element, the equal pair. These belong in the generator, not left to chance. **Grow size across the run.** Start small so early failures are readable, and increase size and depth as the run proceeds so deep structures are reached at least sometimes. **Model the traffic you actually have.** The generator should reflect how the real workload is shaped - repeated hot ids, bursts, retries of the same request - not an imagined uniform world. ### Measure, do not assume This is what separates a senior answer. Two cheap instruments: 1. **Classify and report.** Label each generated input with the classes you care about - batch contains a repeated playlist id; batch is empty; two updates share a version - and print the share of each at the end of the run. Most property libraries offer this, and some let a run fail when a required class stays under a threshold. Seeing *0.02% contained a repeated id* ends the argument in one line. 2. **Check what the run reached.** Run the property suite under a coverage tool and look at the branches inside the code under test. Generated inputs that never reach the conflict branch cannot test it, no matter how many there are. A related trap is over-correcting: constraining the generator so tightly that it only produces the shapes you already imagined. Then the suite tests your assumptions rather than the space. Aim for valid-by-construction plus deliberate skew toward the interesting relationships, while leaving room for shapes you did not think of - the value of generated input is precisely that it produces inputs you would not have written. Finally, distribution problems and shrinking problems are different problems. Shrinking makes a *found* failure readable; it cannot find anything. If the generator never produces two updates to one playlist, no amount of shrinking quality will surface the ordering defect.