A garak scan configured with one generation per prompt reports a probe as passing; the identical scan rerun the next day fails it. What about the tool's sampling explains the flip, and how do you pick a repeat count that makes it unlikely?

**The mechanism.** A garak attempt is one prompt sent once and scored once by that probe's detectors. Each attempt is recorded on its own; nothing in the pipeline collapses the repeats of a prompt into a single per-prompt verdict, and no repeat votes on another. So the per-attempt result is a Bernoulli variable and the scan as a whole is a sampler. Why the variable is not a constant: decoding samples (temperature and nucleus settings on the generator), serving stacks introduce their own nondeterminism through batching and routing, guardrail classifiers sit at a threshold that a small token-level difference can cross, and hosted endpoints roll backend versions underneath a stable model name. Any one of these is enough to make a prompt fail on Tuesday and pass on Wednesday with no code change anywhere. **The arithmetic you should be able to do at a whiteboard.** With per-attempt firing probability *p* and N repeats of that prompt, the probability the run sees nothing is `(1 - p)^N`: ``` p = 0.10: N=1 -> 0.90 miss N=10 -> ~0.35 N=30 -> ~0.04 p = 0.02: N=1 -> 0.98 miss N=10 -> ~0.82 N=30 -> ~0.55 ``` Inverted, the count you need for a tolerated miss chance m is `N = ln(m) / ln(1 - p)`. For a 5% miss chance: about 29 repeats at p = 0.10, and about 149 at p = 0.02. Two things fall out. First, `-g 1` is not a cheap version of the scan; it is a different instrument, one that only reliably detects near-deterministic failures. Second, rare behaviours are brutally expensive to rule out — at p = 0.02 even thirty repeats leaves better-than-even odds of a clean sheet, so "the scan was clean" must never be restated as "the behaviour does not occur". **What it costs to buy that confidence.** The counts above are per prompt, and the bill is per selection. Re-running one 20-prompt probe at `-g 30` is 600 calls — minutes and cents, and you should simply do it whenever a flip matters. Running a 2,000-prompt selection at `-g 30` is 60,000 calls: at a rough 600 tokens per call that is 36M tokens and, at three requests per second sustained, most of a working day of wall clock. That gap between "cheap on one probe" and "unaffordable across the sweep" is the entire reason teams tier the count instead of picking one value for everything. **Where the number misleads.** The headline trap is the clean report. A `-g 1` sweep that comes back with no hits reads, to everyone downstream, as evidence of absence; it is nothing of the sort, and it is the same document six months later when someone cites it in a risk register. The symmetric trap is the single hit: at `-g 1` you cannot distinguish a behaviour that fires on every attempt from one that fires two percent of the time, because both appear as exactly one failed attempt. Those are different engineering problems with different owners and different fix dates, and distinguishing them is precisely what raising the count buys. A third misreading is quieter: comparing yesterday's rate with today's when the counts differed. Two rates from different sample sizes are two different measurements, and the difference between them mixes a real change with a measurement change in a proportion nobody can recover afterwards. **What to check before you call the flip sampling noise.** Configuration drift produces flips that look exactly like randomness and are not. 1. **Same probe selection and same repeat count** in both runs — read both from the runs' own recorded configurations, not from the command you think you typed. 2. **Same generator decoding settings.** A temperature change moves *p* itself. 3. **Same endpoint and model version.** A hosted model name can point at a new backend without notice, and a pinned snapshot is worth having for exactly this reason. 4. **Same detector behaviour.** A garak upgrade between the two runs can change a detector, and the flip is then in the judge, not the target. Only when those four match is "the target samples" the actual explanation. And if it is, the cheap next move is not to re-run the sweep — it is to re-run the single suspect prompt at a high count and measure its firing rate directly, then report the rate with its count attached.