An automated red-team run against a hosted assistant finishes, but roughly 30% of its attempts ended in transport errors, rate-limit rejections or empty responses rather than reaching a scored verdict. How should the coverage statement in the report treat those attempts?

An unscored attempt is dangerous because it is *quiet*. A failed attack and a failed HTTP call both land in the results as a row with no hit, and every harness that computes hits over attempts puts both on the safe side of the fraction. Thirty percent unscored is therefore not a noisier result; it is a rate whose denominator contains 1,800 events that were never judged, and clean areas that may be clean only because nothing arrived. **Where the number comes from, mechanically** `garak` writes one record per attempt into `*.report.jsonl` and copies only detector hits into `*.hitlog.jsonl`, so its per-probe score is hits divided by attempts generated — an attempt whose generator raised is still an attempt that was generated. `promptfoo` distinguishes an `error` result from a `fail`, which is the right shape, but any summary line that reads “N passed of M” has already made a decision about which bucket the errors went to. The first thing to establish about your own harness is which of those two behaviours you have; the answer is in the per-attempt records, not the summary. **Classify the losses before interpreting them** Transport timeouts, connection resets, provider rate-limit rejections (HTTP 429), quota exhaustion, harness parse errors on a malformed response, a policy refusal returned as an empty body, and a multi-turn session that died mid-conversation are all different events wearing one symptom. Only some are yours. An empty body that was really a moderation refusal is a *result* — the target declined — and filing it as an error understates the control; a genuine timeout filed as a refusal overstates it. If the harness does not separate them, the separation has to come from status codes, latencies and any moderation signalling in the raw logs. **The sample you lost is not random** Error rate correlates with exactly the attributes that make an attempt strong: length, encodings that inflate token counts, tool-heavy flows, and long multi-turn campaigns that have more turns in which to die. The lost 30% is weighted towards the attempts most likely to have produced a finding, so the surviving remainder is not a smaller sample of the same distribution — it is a biased one, biased in the direction that flatters the target. **What it costs to fix** Cheap, and the arithmetic is worth doing out loud. Re-driving 1,800 lost attempts against a metered endpoint at roughly a cent of tokens each is tens of dollars; the binding constraint is wall clock, because the fix is to *slow down*. At a sustainable two requests per second with backoff, 1,800 attempts is about fifteen minutes of machine time plus an hour of engineer time to lower concurrency, add retry with jitter, and re-run the lost subset. Compare that with the cost of a wrong clean verdict on an area the client then ships. There is almost no engagement in which re-driving is not the cheaper branch — which is why “we ran out of time” is rarely the real reason it does not happen; nobody looked at the unscored count. **Rate limiting is two claims, not one** The throttle cost you coverage, and it may also be a control a real attacker faces. Those are independent statements needing independent support. “We could not complete X because of provider throttling” is a coverage loss and belongs in the declaration. “The throttle limits an attacker to N attempts per minute” is a finding, and it is only true after you have established whether the limit is per key, per IP or per account, and how cheaply it parallelises. Never let the second sentence retire the first. **What goes in the report** ```text Area attempted scored hits unscored (cause) Direct injection 2,400 2,290 18 110 (429 x86, timeout x24) Indirect via retrieval 1,800 1,020 0 780 (session drop x612, parse x168) PROVISIONAL Tool misuse 1,800 1,740 6 60 (timeout) ``` Three numbers per area, unscored broken out by cause, and an explicit provisional marker anywhere the unscored share is material. Then note which attack families lost the most, because that is the sentence that tells a reader which way the bias runs. **What I check** Does the harness fold errors into passes? Is the printed rate computed over scored rather than attempted? Does the unscored share differ by attack family — and if so, is the family that lost most also the aggressive one? Were empty bodies separated from transport failures using status codes rather than assumption? And has every area with a material unscored share either been re-driven or labelled provisional in the text a reader will actually see?