Red-Team Benchmarks & Datasets

The judge reads attacker-controlled text, so the transcript is untrusted input to it. Payloads can address whatever reads them next and push a scoring decision, and encoded or obfuscated replies fall outside a trained classifier's training data. Hand-adjudicate a random sample of both scored-hit and scored-miss transcripts.

Re-run the frozen benign prompt set against both builds and compare refusal rates. Real hardening leaves the benign rate flat while the attack rate falls; a blanket refusal shift moves both together. Hold the prompt sets, system prompt, decoding settings and labelling rubric identical, and change only the build.

Decide from the claim the number must support. Breadth, many behaviours at few attempts, finds unexpected harm categories and gives a stable per-behaviour picture. Depth, fewer behaviours at many attempts, shows what a determined attacker eventually gets. Most engagements buy a wide cheap pass first, then spend the remainder deep where the pass looked weak.

Almost nothing about the gap. Their 2% may be one attempt per behaviour against your twenty. Treat it as a lower bound, ask for attempts per behaviour, the stopping rule and what ruled a hit, and re-measure yourself at a declared attempt budget before putting the two numbers in one table.

Check that both ran the same frozen prompt set and the same judge, that the gap is bigger than run-to-run variation, and that neither filter was tuned on those exact prompts. Then look at the per-behaviour breakdown and at what each filter costs in wrongly blocked legitimate traffic.

Its prompts are public and old, so vendors have likely patched exactly those strings and they may sit in training or filter data. A low score then proves those specific attempts fail, not that the model resists new ones. Pair it with freshly generated attacks and attacks aimed at your own application.

You changed the measured population, not the model. The mixed figure is comparable to nothing: not to your earlier run on the public list, and not to anything anyone else reports on it. Report the public list and your own behaviours as two results, each with its own count, and never merge them into one headline.

Pull the item-level results for both dimensions and look at what the moderation layer did to each prompt. The usual story: it blocks unsafe completions, so the dimension that rewards refusal rises, while a dimension that penalises unhelpful or over-cautious answers falls on the same behaviour. One change, two scoreboards.

It costs skilled authoring time, labelled ground truth, and a refresh cadence as the set ages. Keeping it clean is policy: never publish examples, only aggregates; send it only to endpoints under no-training terms; hold a sequestered slice nobody iterates against; and keep the tuning team from seeing it.

Spend most of the engagement on cases only your application can fail - its system prompt, tools, retrieval and tenant data - and reserve a thin, repeatable slice of published items as a calibration tripwire. Tell leadership the vendor number is a supplier-selection and regression signal about a component, never a claim about the product you ship.

Tie expiry to events, not just to the calendar: any endpoint or guard change, a dataset or judge revision, or a new attack family invalidates the number. Add a calendar backstop of a quarter or two. Once expired, a published score may inform shortlisting only; anything gating a launch must be reproduced in-house with a recorded manifest.

Treat the judge as part of the metric definition. Pin a versioned judge for the published series, archive every raw completion, and when you upgrade, re-score the whole history and publish both series over an overlap window. Never swap silently; state the judge and its measured error rates.

Report both rates per build with their prompt-set sizes, labelling rules and run configuration, and state the direction each moved. Refuse to combine them into one index, to declare a winner, and to imply the two rates are in the same units. Add the qualitative slice: which legitimate requests the safer build now declines.

For: it is cheap, repeatable and comparable across releases, so a regression is visible. Against: a public frozen suite can be optimised against and says nothing about your own application's attacks. Use it as a non-regression tripwire, and gate release on fresh adversarial runs against your deployed stack.

Both, with different jobs. The shared list is a cheap cross-product regression signal and lets you compare candidate models. It cannot represent product risk, so every product also owns a behaviour register drawn from its own threat model, and that register, not the shared figure, gates release.

Use it as a cheap, repeatable tripwire and as a way to pick where to attack next. Run it early, then on a schedule. Never let a dimension score stand as an assurance claim that the system is safe: it is a fixed, thin, non-adaptive prompt set that knows nothing about your product.

Freeze everything except the model: the pinned item list, the attack method and its budget, attempts per item and the aggregation rule, decoding settings, the target wrapper, and the exact judging procedure. Silent breakers are a refreshed item list, an updated judge, a changed default sampling setting, and the list leaking into training data.

Fix what makes numbers comparable across releases: the behaviour list, a minimum attempts per behaviour, the stopping rule, the hit rule, and a caption that states all of them. Leave extra deep runs and new attacks free, reported separately. Over-fixing costs discovery: a frozen budget stops anyone probing where the model actually looks weak.