Measuring a Guard

It must say what counted as a bypass in the numerator: the guard failed to flag, or the model then actually complied - two different numbers. It must say whether the denominator is attempts, distinct attacks, or attempts per attack. And it must say how many retries each attack got, because retries move the rate without the guard changing.

Every item in the corpus is an attack, so blocking everything means catching everything: the measured block rate hits 100% and nothing penalises refusals. The corpus can only observe one error type. Add benign prompts the product really sees, and report how many of those were wrongly blocked alongside the attack numbers.

The block threshold you ran at, which categories were set to block, the guard's identity and configuration snapshot, and the date. A bypass percentage is only true at one operating point, so without those settings written down nobody can rerun your test or compare a later number to yours.

Multiply three numbers: daily request volume, the estimated share of requests that are attacks, and the 12% bypass rate. Ten million requests a day at one attack per thousand gives ten thousand attacks and about twelve hundred bypasses. Publish the prevalence estimate and its source next to the result.

10% is 400 over attempts, so it partly measures how the corpus was sampled and retried rather than the guard. Report it beside the per-attack rate: how many of the 200 distinct templates got past at least once. If 180 bypassed, the guard is far weaker than 10% suggests; if 4 bypassed 100 times each, far stronger.

Pull benign items from the product's real traffic and from topics sitting right next to the blocked ones: safety questions, clinical or legal wording, security research, fiction, non-English phrasing. A hard negative looks like an attack on the surface but is a request the product must answer. Label them, review the labels, freeze the set.

One threshold collapses the whole score range into a flattering number and hides where the guard actually breaks. A curve shows how bypass changes as the threshold moves, so the reader sees whether the guard is genuinely strong or just parked at a lucky setting, and what a small retune would cost.

Estimate it by hand-labelling a random sample of raw traffic, not from what the guardrail blocked. The block counts include only attacks it caught, so they omit exactly the misses you are trying to size, biasing prevalence low. Report the estimate as a range with the sample size that produced it.

Per million requests: 1,000 attacks produce 900 flags, and 999,000 benign prompts produce about 9,990 flags. So roughly 10,890 flags contain 900 real attacks, about 8%. Around eleven in twelve flags are benign. Report that share, because a rate measured on an all-attack corpus never reveals it.

Fix the attempt budget and count templates that bypass within it - say, the share of the 200 that got through in 10 attempts or fewer. Better, record attempts-to-first-bypass per template and report its distribution. Both runs must use the same budget, so you re-run the cheaper one. The cost is queries, money and wall-clock against a metered endpoint.

Count an over-block whenever a benign request fails to get the answer it deserved: a hard refusal, a hedged non-answer, or a silently rewritten reply. One overall percentage hides which kinds of user are hit, so break it out by benign category and hand over the actual blocked requests as examples.

It means the guard barely separates those attacks from allowed traffic: the blocks are luck of the line, not margin. A small loosening, a guard update, or a slightly reworded attack flips many of them through. Report the margin distribution and how bypass moves for small threshold changes, not just the current number.

Sweep whatever the service does expose instead of a score: each selectable strictness level and each category toggle. Run the same corpus at every setting and report a step curve over settings rather than a smooth score curve. Pin each configuration exactly, and state that the resolution is limited to the settings offered.

Prevalence is partly chosen by attackers, so it is not a fixed property of your traffic. A targeted attacker sends 100% attacks and experiences the raw conditional rate; a campaign can raise prevalence overnight. Present both the ambient restatement and the targeted case, and state the prevalence assumption on the same page.

Split the corpus. Freeze a versioned baseline slice and report the bypass rate over that fixed set of distinct attacks at a fixed attempt budget - that number alone is compared quarter to quarter. Report newly added templates as a separate first-run number, never blended in. Blending makes the trend move when the corpus changes rather than the guard.

Spend enough on benign traffic to detect an over-block rate the product would actually care about, then give the rest to attacks. Size the benign half from the smallest rate worth acting on, not from what is left over. Defend it by noting a catch-rate-only report cannot tell a good guard from a closed door.

Hand over a function, not a constant: bypass across the threshold range, the operating point it was measured at, and the raw per-attempt scores so any future setting can be re-derived. Fix the operating point before results are seen, and make a threshold change trigger a re-measurement rather than a reinterpretation of your number.