A published red-team benchmark delivers every item as a user turn, but your deployment also puts text you did not write - retrieved documents and tool results - into the same context. What do you re-run against that second channel, and how do you keep it a bounded job instead of a second full benchmark?

**The shape of the problem.** A published suite has exactly one delivery channel, because a chat-completions endpoint has exactly one: the item goes in as the user's message. A deployment normally has three or more — the user turn, retrieved context, and tool or connector output — and the model sees them in a single window. So "we ran the suite" is a coverage claim over one column of an item-by-channel matrix, presented as though it covered the deployment. That denominator swap is the whole defect. **Selection, which is what keeps the job finite.** The instinct is a cross-product: every item on every channel. Do not. Port an item to a non-user channel only where the channel plausibly changes the outcome, and the useful filter is: *does this item's success depend on the model treating supplied text as an instruction, or on the model producing content it should withhold?* The first kind ports, because "who authored this text" is precisely the variable that changes. The second kind mostly does not — a knowledge request that the model refuses on the user turn refuses much the same when the same words arrive from a document. In practice that leaves a small fraction of the corpus, perhaps a few dozen items, and that is the intended outcome: a matrix you can actually re-run each release beats a cross-product you run once and never repeat. **Harness work, at the level of the object that does it.** You need three things per ported item. A **fixture**: the item text placed in a document that your indexer will accept, or in a stubbed tool result your orchestrator will return. A **seeding step**: writing that document into the retrieval index used by the test tenant, then confirming the index has committed it — most vector stores index asynchronously, and a run started too early tests an empty index. And a **trigger turn**: a benign user question whose embedding actually retrieves the fixture. That last one is where arms die silently, because your retriever returns top-k by similarity and your fixture may simply never be in the top-k. **What it costs.** Per ported item, expect fixture authoring plus a trigger turn plus an index write — roughly 15-30 minutes of engineer time each the first time, so forty items is one to two engineer-days before a single call is made. The run itself is cheap by comparison: forty items times two channels times three samples is 240 application calls and 240 judge calls. The expensive standing cost is the sandbox. Tools stubbed as no-ops make the entire arm meaningless, because the outcome you are hunting on this channel is usually an action; a sandbox faithful enough for an action to register, and safe enough to run repeatedly, is the real budget line. Index rebuilds and tenant isolation add operational time each run. **Where the number misleads.** - **The zero that is a no-op.** An arm reporting zero hits is the most dangerous result in this whole exercise, because "the app resisted every injected item" and "the fixture was never retrieved into the context" look identical in the aggregate. Without a per-item retrieval hit/miss record you cannot tell them apart, and the comforting reading is the wrong one more often than not. - **The merged percentage.** Reporting user-channel and retrieval-channel results under one figure lets a thin, deliberately sampled arm inherit the credibility of a full sweep. - **The suite's denominator.** "100% of the published suite" uses the benchmark's item count as the denominator for a deployment whose surface is a matrix. Coverage here is cells attempted over cells selected, and the selection rule must be stated. - **Prose-only judging.** On this channel the consequential outcome is often a tool call, so a text judge reports a clean sweep exactly where the risk lives. **What you check.** Per-item retrieval hit/miss, before you read any rate at all. Whether the fixture was indexed and committed before the run started. Whether tool calls are in the judged unit. Whether the selection rule is written down, so a reader can see the arm was sampled deliberately and thinly rather than swept. And two denominators in the report, never one. The attack techniques that exploit this channel, and the design of an agent harness that drives it, belong to the neighbouring topics; the job here is the instrument's coverage bookkeeping.