You wrote a new probe for the garak LLM scanner and pointed it at a detector that ships with the tool for a different attack family. Why is the failure rate that comes back untrustworthy?

### What a shipped detector actually is In garak a detector is a small object with one job: given an `Attempt` — a prompt and the target's replies to it — return a score per reply, which the evaluator thresholds into hit or pass. The implementations are deliberately cheap. Some are substring matchers configured with a list of strings and a match mode (anywhere in the reply, whole word, at the start). Some apply regular expressions. Some wrap a small off-the-shelf text classifier — a toxicity model, a refusal classifier — and emit its probability. A few invert the test and score a *missing* refusal as the failure. Every one of those was fitted beside a particular probe. The string list is the wording that family's failures actually produce. The classifier was chosen because its label space matches that family's harm. The inversion is correct because for that family, compliance is the failure. That fitting is precisely the part that does not travel. ### The two errors you inherit at once Point your own probe at somebody else's detector and both error directions go live simultaneously. **Misses.** Your family's genuine failure replies do not contain the strings, match the patterns, or land in the class the borrowed criterion looks for. Real hits score clean, and they vanish — a miss leaves no artefact anywhere in the report. **False positives.** The criterion is loose enough to fire on wording your prompts happen to induce. A substring detector for a refusal phrase will fire on any reply that quotes or discusses that phrase. A toxicity classifier will fire on a reply that describes the harm rather than committing it. Benign replies score as hits. Because both are live, you cannot even say which way the rate is wrong. The honest description of a borrowed-detector rate is not "conservative" and not "inflated" — it is *undirected*, and it stays undirected until somebody reads transcripts. | what you get | what it looks like in the report | how you find it | |---|---|---| | miss | nothing at all | read the replies the detector cleared | | false positive | a hit line like any other | read the replies it flagged | | both together | an ordinary-looking percentage | read both sides, then reweight | ### What the mismatch costs The compute cost of the mistake is one wasted run — the same prompts x generations bill you already paid, spent again after you find out. The larger cost is triage: a 1,200-output run read end to end at fifteen seconds per reply is about five hours of somebody's attention. The affordable version is a stratified sample of roughly fifty flagged and fifty cleared replies, which is around half an hour and catches a gross mismatch immediately. The worst cost is reputational, and it is not recoverable by re-running: a number that reached a slide and then moved by ten points when the detector was fixed teaches the audience to discount everything the team publishes. ### Where the number misleads Nothing on the page marks a borrowed ruler. A rate is a rate; the report renders your custom pairing exactly like a shipped one. Downstream that mismatch propagates: if the pairing is quoted against garak's calibrated positioning for the shipped family, you are comparing your prompts against reference data collected with different prompts. And over time it corrupts trend. Re-run next release, watch the rate move, and the natural story is "the model changed" — when a detector whose criterion never fitted will drift with any change in the target's formatting, refusal style, or verbosity. ### What I would check before believing it Write the failure signature of my family as one sentence a colleague could apply by hand without reading code. Then ask whether the borrowed detector's criterion *is* that sentence — not adjacent to it, not in the same area. If I cannot state what reply property the detector fires on, I do not know what I measured. Then the cheap empirical check, before spending a full run: call the detector offline on a handful of replies I have already labelled — some obviously failures for my family, some obviously benign. If it cannot fire on the ones I wrote to make it fire, the mismatch is proven for the price of a unit test. Only after that does a stratified transcript sample from the real run mean anything, and only then does a rate belong in a report — quoted with the sample, the criterion and the measured error beside it. "Close enough" is exactly the condition under which the number is confidently wrong.