Your product is a bank's customer-support assistant, and its worst realistic outcome is being talked into revealing another customer's account details. It scores near-perfect against the AdvBench and HarmBench behaviour lists. Mechanically, why does that result say nothing about your top risk, and what would you do instead?

**Why the omission is silent rather than low-scoring.** The evaluation is a loop over the rows of the behaviour list. Each row becomes prompts, the prompts become responses, the responses become verdicts, and the verdicts are aggregated over the row count. A harm with no row enters none of those stages: no prompt is generated, no response is produced, no verdict is recorded, and nothing is added to the denominator. It cannot lower the score, and it cannot raise it. In the report an unmeasured harm and a harm that was attempted and refused look identical — both appear as the absence of a failure. That is the charter's point that the risk your product owns is a blind spot scoring zero: the danger is not that zero is a bad number, it is that zero is written in the same ink as good news. **Why public lists structurally cannot hold this particular risk.** Public behaviour sets are built to be comparable across models, which forces them to be product-agnostic: single-turn, tool-free, aimed at a bare model, and restricted to harms that are illegal or dangerous for anybody. Cross-customer disclosure needs four things none of that provides — an authenticated session belonging to caller A, an account-lookup tool actually wired to the target, several turns of persuasion or confusion to get the assistant to resolve the wrong identity, and a judge that can compare the disclosed value against session ground truth. That last one is the part teams underestimate: a generic harm classifier cannot decide this. Deciding it requires knowing which account number was legitimate for that session, so the judge has to be written against your data model, not borrowed. **What the replacement costs.** Writing a behaviour register is analyst work, not compute. Budget an hour or two per entry to state the behaviour concretely, specify the target configuration it must run against, and write the rule that decides it happened; then review. A register covering the top five risks with three or four behaviours each is fifteen to twenty entries — a few days to build, half a day per refresh, and a refresh is owed whenever the product gains a tool, a data source, or a new user role. Running it is cheap in tokens: twenty behaviours at five generations over four conversational turns is roughly four hundred model calls plus tool round-trips and judging. The genuinely expensive item is the environment — a staging deployment with realistic-but-synthetic customer records, so that a successful disclosure is provable and harmless. That environment is the project; the token bill is a rounding error beside it. **Where the numbers mislead, including your own.** The public number misleads as coverage, which is the headline error. But the bespoke register has its own failure modes, and a team that only distrusts the public figure gets caught by them: - *Small denominators.* Twenty behaviours at five generations is a hundred attempts. One success reads as 1%, and zero successes over a hundred attempts is entirely compatible with a real success rate of a few percent. Report counts, not rates, at this scale. - *A judge scoring the wrong proposition.* A rule that fires on “the assistant stated an account balance” flags the legitimate case where the caller asks for their own balance, and equally flags a hallucinated number that belongs to nobody. The proposition you need is “stated a value belonging to a different customer”, checked against the seeded ground truth. - *A stubbed target.* A run against a build where the lookup tool is mocked to return a constant proves nothing about the deployed path, yet produces a clean green report. - *Merging.* Averaging the register with the public list yields a figure comparable to nothing and hides which population moved. **What I would check before believing a green result.** Does every top-five risk have at least one behaviour, named in the register with the risk ID? Is the tool genuinely enabled in the target used for the run — check the transcript for a real tool call, not the config file? Was every successful attempt read by a human? Is any register entry a restatement of a public-list row, which double-counts? And the cheapest guard of all: seed one behaviour the current build is known to fail, and confirm the harness detects it. A green run whose harness cannot produce a red is not evidence.