Machine-drafted cases triple a regression suite's case count in a week. What has that growth not proven?

## What a case count counts A suite's **case count** is a count of artefacts: files, functions, rows in a plan. What a suite is *worth* is a different kind of object entirely - a **set**, namely the set of distinct wrong behaviours that would make at least one case fail. Call that set the suite's **reach**. Every case contributes two things to reach: the parts of the system its run causes to execute, and the outcomes it actually checks about that execution. The suite's reach is the **union** of those contributions across all cases, not their sum. A case whose executed parts and checked outcomes both already sit inside that union is adding a member to a set that already contains it. The count goes up by one. The reach goes up by nothing. That is the whole mechanism of a coverage illusion under bulk drafting, and it fits in one line: **counts add, reach unions.** ## Why drafted volume repeats itself Redundancy here is not carelessness; it is what the setup produces by default. - Drafts are made from the same sources the existing cases were made from - the same requirement text, the same interface description, the same code - so they rediscover the same journeys through the system. - The variation a drafting pass produces most cheaply is surface variation: different names, different step ordering, different sample values drawn from **inside one equivalence class**. - The drafting step usually has no picture of what the suite already reaches, so it cannot aim at the gap. Without that, heavy overlap with the existing suite is the expected outcome rather than a bad day. - Volume arrives faster than review capacity, so the moment where somebody once said "we already have that one" simply does not happen for most of a batch. - Where a description is ambiguous, a drafting pass tends to produce several cases for the *same* reading of it, rather than one case per reading - the opposite of what the ambiguity called for. ## Three numbers, and which of them can move | Number | Moves only when | Under a redundant batch | | --- | --- | --- | | Case count | any case at all is added | rises by the full size of the batch | | Structural coverage | code that no case executed now runs | flat, or drifts slightly on setup paths | | Reach | a new outcome is checked on behaviour the suite exercises | unchanged | The middle row is the one that fools people in both directions. A structural figure can only rise when some part of the system that nothing reached is now reached, so adding traffic to already-executed code cannot move it: a batch that leaves the figure flat has genuinely told you something. But a batch that *does* move it has told you only that something new **ran**. Reach needs the second ingredient as well - an outcome checked - before a wrong behaviour in that newly executed code could fail anything. ## Reading a batch honestly The honest read of a drafted batch is its **marginal contribution**, and getting it is mechanical: 1. Before the batch, record the union of executed units across the suite and the union of asserted outcomes. 2. Run with the batch and record both unions again, then subtract. 3. Mark every case whose executed set and asserted set both fall inside the old unions as **count only**. 4. For each case that remains, name the wrong behaviour it would catch, and check whether an existing case already catches it. The output is one small honest sentence. Not "200 cases added", but "six behaviours became reachable, four of which were already checked elsewhere". That sentence is what the batch actually did, and it is usually a great deal shorter than the batch. ## What the inflation costs Cases are cheap to produce and are never cheap to keep. - **Run time.** Every redundant case is paid for on every run, for as long as it exists, in wall-clock time and in machine capacity. - **Triage width.** One defect sitting under forty redundant cases produces forty failures to read, and forty failures look like a large problem rather than a single one. - **Reading cost.** Anyone changing the code beneath those cases must first work out what each of them claims, and forty near-identical claims take far longer to understand than one. - **False confidence.** The dangerous cost is the belief that an area is well covered because many cases mention it, because that belief stops the work that would actually have closed the gap. ## When volume genuinely pays Bulk drafting is not the problem; **unaimed** bulk drafting is. Point the same capacity at a gap that has already been identified - a list of branches nothing executes, an inventory of error paths nobody checks, the input classes a stated rule defines but no case supplies - and every draft lands outside the union by construction. Then count and reach rise together, which is the only condition under which the count was ever worth quoting.