Replacing a contaminated public attack corpus means building and maintaining a held-out attack set of your own. What does that cost an organisation over time, and what rules stop the replacement from becoming contaminated too?

Treat a held-out attack set as an asset with a depreciation schedule, not as a file that someone checks in once. ## Build cost - **Authoring.** The people writing it must know the attack families well enough to cover the *behaviour* space rather than the *phrasing* space. Rewriting a public corpus with synonym substitutions is the cheap trap: contamination attaches to phrasing patterns as much as to exact strings, so a derived set inherits the very surface cues you were trying to escape and quietly measures the same memorisation. Realistic figure: one to two engineer-weeks of skilled time for a few hundred usable items, and that number recurs every refresh. - **Ground truth.** Every item needs a written definition of what a successful response looks like, at the granularity your scorer can actually apply. Ambiguous items generate judge disagreement that later gets misread as model drift. - **Scoring.** You still need something to grade responses. If you grade with a hosted moderation classifier you have imported someone else's blind spots — including, plausibly, training on the same public safety corpora you left behind. Budget human adjudication of a sample every cycle to calibrate whatever automated scorer you use. Note what is *not* expensive: the runs. A few hundred prompts at a handful of samples each, plus one grader call per completion, is a few thousand short calls — single-digit to low-tens of dollars, under an hour of wall clock, and a scanner such as garak scales exactly linearly through its `--generations` flag. Anyone who budgets this work by API spend will conclude it is nearly free and will not fund the part that actually costs money. ## Depreciation The set loses value through every exposure: prompts sent to endpoints that retain inputs, examples pasted into tickets and slide decks, and — the largest by far — the internal team optimising against it. Every other leak is slow; a tuning team with access contaminates the set within one cycle, because a private eval that is also a target is just a public benchmark with a smaller audience. Plan a refresh fraction per cycle and a full rotation horizon. ## Hygiene rules worth writing down 1. Reports carry aggregate rates and behaviour-class names, never prompt text. 2. Any endpoint without contractual no-training and retention limits is treated as **publication**; prompts sent there move to a burned pool and are rotated out. 3. A sequestered slice is never used during iteration. It is opened only for the release read, and once a decision has been argued against it, it rotates. 4. An access boundary separates measurement from tuning. 5. Provenance is recorded per item: author, date, behaviour class, and whether it has ever left the boundary. ## Where the private number misleads A held-out rate feels authoritative because you built the set, and that feeling is the risk. Four specific misreadings: - **"It is private, therefore it is clean."** Only until someone tunes against it. Privacy is a property of the process, not of the file. - **Trend read across refreshes.** When you rotate items, this quarter's rate is measured on a partly different set. A movement can be composition change rather than model change; keep an anchor slice unchanged across at least two cycles so you can separate the two. - **Cross-vendor comparison.** A private rate is not comparable with anyone's published number and cannot be independently reproduced, which is a real cost when a regulator or customer asks for evidence. That is the argument for keeping a public corpus in the report as a labelled, comparable, contaminated line. - **Coverage mistaken for robustness.** A low rate on a set that covers four behaviour classes says nothing about the fifth. ## What you check each cycle Re-run a small anchor slice unchanged and confirm its rate is stable, so a moving headline can be attributed to new items rather than to the endpoint or the scorer. Adjudicate a human-labelled sample and record the automated scorer's agreement rate; a drop there is scorer drift, not model change. Audit exposure: how many items left the boundary this cycle, and were they rotated. And compare rates on reused versus fresh items — if reused items score systematically better, the set has begun to leak into whatever the tuning team is doing. ## When not to build one If the organisation cannot fund authoring, adjudication and rotation, a half-maintained held-out set that everyone quietly tunes against is **worse** than an honestly caveated public number, because it looks trustworthy and is not. That judgement — whether the recurring budget exists — is the actual decision here, and it is made once a year, not once.