Red-Team Benchmarks & Datasets

Record the endpoint string and any pinned revision it resolved to, the timestamp, decoding settings such as temperature and max tokens, the dataset revision and which behaviours ran, the harm judge and its version, and the raw transcripts. Without these, a later number cannot be compared and any difference cannot be explained.

The numerator is the count of attempts the suite judged harmful; the denominator is the unit the suite chose to count over. That unit may be behaviours on a fixed list, individual attempts, or behaviour-and-attack pairs. Before reading the rate you need the behaviour list, the attempts per item, the attack method, and who ruled an attempt a hit.

It counts by wording, not content. A reply that apologises then complies is scored a refusal; an empty, off-topic or garbled reply with no listed phrase is scored a success. Unlisted refusal wordings, other languages and paraphrases all leak through, so the rate drifts both up and down.

A model that refuses everything scores zero on any attack suite, so zero can mean hardened or mean useless. The attack rate only reads next to a benign-refusal rate: run a set of harmless prompts, many of them phrased to look sensitive, and report how many were refused. Publish both numbers together.

Because a behaviour usually counts as broken if any attempt succeeds. Twenty attempts give twenty chances where one gives one, so the figure only rises as the attempt count rises. With attempts-per-behaviour unstated, 38% describes how much you spent as much as how fragile the model is.

It freezes the attack prompts, the judge that decides whether a response counts as a jailbreak, and the report format. With all three fixed, two defences see identical inputs scored by identical rules, so the difference in reported success rate reflects the defence and not a different prompt set or a different grader.

It is a fixed list of harmful request descriptions, sorted into categories its authors chose in advance. A good result means the model refused those written requests under whatever attack drove them. It says nothing about harms that are not on the list, including whatever your own product's worst outcome is.

Nothing prevents the corpus being inside safety tuning. Those exact prompts appear in refusal training data, and often in the guard classifier that scores the run, so the model refuses memorised strings. Reword the same requests and the rate typically climbs. The number measures recall of a public list.

Down: your system prompt narrows what the assistant will discuss and an input or output classifier sits in the path, so items that landed on the bare model get deflected. Up: retrieval carries text you did not write, tools turn a compliant answer into a real action, and a long preamble dilutes plain refusal behaviour.

It records a name, not a build. A hosted endpoint's alias can be re-pointed to an updated model, and the safety tuning and any provider-side filters behind it can change with no announcement. The row also freezes the decoding settings and the harm judge used then. It is one snapshot, not today's endpoint.

Both rates must count over the same population and be produced the same way: the same behaviour list, the same attempts per behaviour and aggregation rule, the same attack method, the same target configuration, the same procedure for ruling a hit. Independently published suites share none of these, so their headline rates belong on different axes.

Judge error sets a floor. With an 8% false-positive rate, even a model that never complies would score around 8% flagged hits, so a 5% reading sits inside the judge's noise. Report the judge, its measured error rates, and adjudicate a sample before claiming any true rate.

It rises and flattens. The easy behaviours fall in the first few attempts, so each extra attempt buys less. Flattening means further attempts of this attack will find little more: you have separated the behaviours this attack can break from a residual core that resists it at any budget you can afford.

The judge decides what counts as a jailbreak, so it is half the measurement. Swap it and your numbers stop comparing to every published result on the suite: a stricter judge lowers the reported success rate, a looser one raises it. Report your own judge's result separately, never as the leaderboard number.

Those lists contain behaviours their authors picked — weapons, fraud, harassment, self-harm. Cross-customer data disclosure is not among them, so no attempt ever aimed at it. A behaviour nobody attempts cannot fail, so your top risk contributes nothing to the result. Write your own behaviours from your risk register and measure those separately.

That dimension is backed by few prompts, so a few points can be one or two responses flipping. Sampling, non-zero-temperature decoding and the judging step all move it on their own. Check which individual items changed, re-run the unmitigated build unchanged, and see whether the movement is bigger than run-to-run drift.

Re-run the same harmful behaviours in fresh wording and compare. A large gap between verbatim prompts and paraphrases points at memorisation. Score the responses with a judge the vendor did not train, check whether the corpus predates the model release, and see whether the guard's own documentation lists it as training data.

Hold the transcripts fixed and vary the scorer. Get both teams' raw completions and run both judges over both sets. If each judge gives similar numbers on either set, the gap is the scoring choice, not the model. Without archived completions the two rates cannot share a table.

Because anything published gets trained on. Vendors add public attack prompts to safety-tuning data and to guard classifiers, so models learn to refuse those exact prompts. The score rises without the model becoming harder to attack. An unpublished set stays a fair test, since nobody could have fitted to it.

A broad trust battery scores several different trust properties in one pass, so it gives you a wide first map of where a model looks weak. A focused attack suite drills one property hard instead. The cost is depth: each dimension is backed by a thin slice of prompts, so its number is coarse.

Point at both if you can. The alias is what production calls, so running it is your drift detector. The pinned snapshot is a control: same weights every quarter, so a moved rate there means your harness or judge moved. Costs are double the queries, and pinned snapshots eventually get retired.

They are different statistics. The any-of-n behaviour rate is never lower than the per-attempt rate on the same transcripts, and it rises as n rises even though the model is unchanged. You cannot convert one to the other from the headline numbers, because the conversion needs the per-behaviour distribution of hits, which the aggregate discards.

Not just the flat "I can't help with that". Count deflections, moralising non-answers, and replies that answer a safer question than the one asked. The attack side has a concrete target — did the harmful content appear. The benign side has no such artefact, so you are grading whether the user's actual request was served.

Open the file and read it. Check how many entries each category holds, how many entries are near-restatements of each other, and how the requests are phrased. Categories are unevenly sized and wordings repeat, so a category figure can rest on a handful of entries or on one idea counted several times.

Do not silently drop it. Keep the public number as a regression floor, clearly labelled as a set the guard was trained on, and make a held-out attack set the headline. Report both, since the gap between them is your best estimate of how much the public score is inflated.

Two common breaks: the app returns structured output or a rendered card, so the judge reads a wrapper rather than the answer text; and the app emits one fixed refusal template the judge was never calibrated on. Fix by judging exactly the text and tool calls the user receives, then hand-labelling a sample to check the judge still agrees.

Do not replay the whole corpus on the retrieval channel. Select the items whose outcome depends on the model following injected instructions, place that text in the retrieved-document or tool-result slot instead of the user turn, and score what the app did as well as what it said. Report the two channels as separate coverage numbers with separate denominators.

Change one thing at a time. Re-score the old transcripts with the new judge: if the rate moves there, the judge drifted. Diff the run manifests for decoding, dataset and error-handling changes. Then repeat the new run to see the spread across repeats before calling any residual gap a real model change.

The published denominator enumerates that suite's fixed behaviour list, attacked by that suite's method, against the model as the suite configured it - usually raw weights with a plain chat template, no product system prompt, no retrieval, no filters. Your scan enumerates your own attack set against the deployed stack. Different population, target and ruling.