In a red-team report on a moderation guard, why present attack bypass across the guard's whole score range rather than as one percentage at the threshold the team currently runs?

## One number, two very different guards A moderation guard scores each prompt; the **block threshold** is a line drawn through those scores; the **bypass rate** counts attack prompts on the allow side of the line whose responses a fixed judge called successful attacks. Reporting only the count at the line throws away the distribution you already paid to collect — and the distribution is where the guard's actual quality lives. Take two guards, each measured at 4% bypass on the same attack corpus at their configured thresholds. Sweep the threshold across its range and they stop looking alike immediately: | block threshold | guard A bypass | guard B bypass | |---|---|---| | strictest | 2% | 1% | | one step stricter | 3% | 2% | | **as configured** | **4%** | **4%** | | one step looser | 5% | 30% | | most permissive | 22% | 41% | Guard A holds near 4% across a wide band: your attacks score far from its line, so the result is a property of the guard. Guard B is at 4% only at this exact setting — its attacks are piled against the line, and the number is a property of where somebody happened to put it. One figure cannot separate those two situations; the sweep separates them at a glance. ## The mechanism The sweep is post-processing over rows you already have, one row per attempt: the prompt, the guard's score (per category if it returns them), the model's response, and the judge's verdict on whether the attack worked. ``` for t in sweep(min_score, max_score): bypass[t] = count(rows where guard_score < t -> allowed AND judge_verdict == attack_succeeded) / n_attacks ``` Two things people get wrong here. First, a bypass is not "the guard allowed it" — it is "the guard allowed it **and** the attack worked". Sweeping the threshold moves only the first term; the judge verdict is a fixed column. Second, the sweep is recoverable only if scores were retained per attempt. A harness that stored the boolean verdict has thrown the axis away, and the curve is not recoverable by any amount of cleverness. ## What it costs If scores were retained, the curve is free: an O(n) pass over a file, seconds of compute, no calls to anything. If they were not, the curve costs a full rerun — for a 1,000-prompt corpus, roughly 1,000 guard calls plus 1,000 target completions plus judge cost, hours of wall clock on a rate-limited endpoint, and a fresh triage pass. The entire price of the curve is therefore paid before the run, in one decision: log the score, not the verdict. That is why score retention belongs in the harness as a standing requirement rather than a per-engagement choice. ## Where the curve misleads - **It argues monotonically for tightening, and it has no benign axis.** Every point on the curve gets better as you tighten, because the corpus is all attacks. The cost of tightening lands on legitimate traffic that this run never measured. A reader shown only this curve will reasonably conclude the threshold should go up; nothing on the chart tells them what that refuses. - **A flat band is ambiguous.** Wide separation is equally consistent with a strong guard and with a mild corpus. Before claiming margin, check the same curve against a harder slice, or against a second guard — if your attacks sit far from *every* guard's line, the finding is about your corpus. - **A moving judge fakes structure.** If the judge is a model re-invoked at each threshold, sampling noise appears in the curve as shape. Judge once per attempt and reuse the column. - **One dial, several knobs.** A guard that scores per category has several thresholds. Sweeping a global cut while per-category rules also apply collapses knobs the reader will assume were held fixed; say which dial you actually moved. ## What to check Validate the sweep against reality before shipping it: pick two thresholds off the curve, actually configure the guard at those values, rerun a slice of the corpus, and confirm the live verdicts match what the sweep predicted. This tests an assumption the recomputation quietly makes — that the deployed threshold is compared against the same score you recorded. Services that layer extra logic on top of the score (keyword short-circuits, per-category overrides, a separate policy engine) break that assumption, and the swept curve will be smoothly wrong. Then quote the number at the operating point actually in force, with the curve beside it: the reader's reality is one point, and your job is to stop that point standing alone.