An automated jailbreak search has produced three thousand prompts its judge marked successful, but they are heavy rewrites of two underlying shapes. Why is the raw count of successful prompts a poor signal for whether to keep spending the remaining queries?

## What the search is doing when it "succeeds" An automated jailbreak search is a loop with three parts. A **generator** produces a candidate prompt; a **target** — the model under test, reached through a metered API or loaded locally — answers it; a **scoring step**, called a judge in some tools and a scorer or detector in others, decides whether that answer counts as a success. Whatever the family, the loop keeps the candidates that scored well and builds the next batch out of them. A token-level optimiser edits a suffix under a numeric signal. An attacker-model loop hands a second language model the last refusal and asks it to rewrite the request. A mutation loop paraphrases and recombines entries in a pool of hand-written framings. All three are hill-climbers, and their entire design is to stay near whatever last worked. That design fixes the shape of the output. Once the loop finds one working shape, the cheapest way to score again is to perturb that shape, so children of the hit arrive in volume and keep arriving. Three thousand accepted prompts descended from two parents is not three thousand results. It is two results, plus a measurement of how reproducible each one is — which is worth knowing, and is a completely different quantity from how much the run discovered. ## What the run costs, in the units you are billed in Queries, not prompts, are the meter, and one *attempt* is usually more than one billed model call. A judged attacker-model loop calls three models per attempt: the attacker that writes the candidate, the target that answers it, and the judge that scores the answer. A campaign described as "ten thousand attempts" is therefore on the order of thirty thousand completions, and a multi-turn loop multiplies the target and judge legs again by turn depth while the target's prompt leg grows with the conversation it is carrying. Against a commercial endpoint every one of those legs is priced per token. Wall-clock is normally set by a tokens-per-minute or requests-per-minute ceiling rather than by compute, so "run it longer" often means "wait longer", not "spend more machine". And the human cost lands after the run: reading three thousand accepted transcripts to discover there were two mechanisms is a day of engineer time that a live cluster curve would have saved. The reason to state all this is that the marginal query has a price, so "it is still finding things" has to be weighed against something rather than treated as self-evidently good. ## Where the number misleads **Budget.** An operator watching successes per hour sees a healthy rate and lets the allowance drain. The rate really is healthy — it is measuring reproduction of a hit already recorded, and each further query buys a duplicate. **Breadth.** "Three thousand successful jailbreaks" reads to any downstream audience as wide coverage of the target's failure modes. The run has deep coverage of two. That gap is exactly where a report turns into a wrong decision in either direction: it can alarm a team about volume that is one weakness, or it can imply a thoroughness the search never had. **Comparison.** Two runs cannot be ranked by success count at all. A run that finds one highly reproducible shape will out-count a run that finds six fragile ones, and the second run is the better result. Any leaderboard built on raw hits pays teams to exploit rather than explore. **The judge underneath.** If the scoring step accepts on a surface cue — a compliance-shaped opening, the mere absence of a refusal phrase — rather than on the behaviour the objective named, false accepts enter the count. Those false accepts are strongly self-similar, so they inflate the total *and* make the run look like clean exploitation of one basin at the same time. A count that is wrong while the curve looks explicable is the worst of the four cases, because nothing in the dashboard flags it. ## What to check - Cluster the accepted prompts and plot **distinct clusters against queries spent**, updated live rather than at the end. The level of that curve is the result; its derivative is the stop signal. - Read the cluster-size distribution. One enormous cluster with a tail of singletons usually indicts the grouping rule rather than describing the target. - Hand-audit a sample of accepted successes: does the response actually contain the behaviour the objective named? Do this before believing anything computed downstream of the judge. - State your denominator out loud. "Hit rate" is not a stable field name across red-team tooling and different tools headline different quantities; say whether you mean accepted attempts over attempts made, or distinct templates over queries spent.