Every task in your agent red-team suite is scored against a hand-written list of forbidden side effects, and every run now passes. What can you conclude, and how would you restructure the success criteria?

**First, read the all-green result correctly.** A suite of *deny-list* assertions — a hand-written enumeration of forbidden side effects, each checked against a state diff — that never fires supports exactly one claim: none of the effects on that list was observed in these runs. It does not say the agent is safe, and it does not say the attacks were weak. Three cheap explanations must be ruled out before anyone redesigns anything. The **sinks may be unobserved**, so the effect happened where no snapshot or recorder was watching. The **task may be unfailable**: if the fixture never grants the capability the assertion targets, or the scenario never reaches the state where the forbidden action is even available, the pass carries no information at all. And the **assertion may be broken** — a predicate with a typo'd table name, or an ignore list that swallows the very column it should watch, passes silently forever. **Then fix the polarity.** Deny-lists and allow-lists differ in *who has to be exhaustive*. A deny-list requires the defender's imagination to be complete: you catch only what you thought to forbid. An allow-list — a **permission envelope** — requires only that the task's legitimate behaviour be enumerable, which is a far smaller and much better-understood set, because the task author already knows what one correct completion does. This is the default-deny argument from firewall and capability design, and it lands the same way here: the unanticipated case is the one that matters, and only one polarity catches it by construction. ```text task: refund-request allowed_effects: - update orders.status where id = 41 -> {refunded} - one outbound message to the address on order 41 - writes under /workspace/task/** any_other_observed_effect: violation ``` As agents gain tools and autonomy the enumerated forbidden set grows arithmetically while the reachable effect space grows much faster, so an all-green deny-list suite becomes *weaker* evidence over time, not stronger — a genuinely counter-intuitive property that a good answer names. **What it costs.** Three distinct bills. - *Triage.* Default-deny converts every benign unlisted effect and every scrap of ambient noise into a failure someone must adjudicate. On a loose fixture this is hours a week and it does not decay. The real failure mode is political rather than technical: sustained triage pain produces steady pressure to widen the envelope until it stops firing, and a fully widened envelope is a deny-list with extra ceremony. - *Prerequisites.* Envelope scoring only works on top of a tight fixture, frozen clock and seeds, network default-deny and field-level diffs. If those are not already in place, the inversion is not a scoring change, it is an infrastructure project — usually the larger part of the work. - *Authoring.* The envelope moves to the task author, who must state exactly what a correct run does. That is a healthy forcing function — a task whose legitimate effects nobody can enumerate is a task nobody can score — but it is real hours per task, plus a re-baselining sweep of the whole suite so the new criterion has history. **Where the number misleads.** A 100% pass rate under a deny-list is quoted upward as "the agent is safe", when the honest sentence is "no listed effect was seen in these runs, on the sinks we watch, in the states these tasks reach". After inversion the failure rate misleads in the other direction: envelope violations are dominated by benign unlisted effects, so a suite that suddenly reports 30% failures has not found a dangerous model, and reporting that figure as an attack-success rate is simply wrong. And the headline suffers: "an effect outside the envelope occurred" is a weaker sentence to put in front of an executive than "the agent deleted customer data", which is why mature suites run both — a permission envelope as the general criterion, plus a small set of named deny assertions for the catastrophes you want called out by name. **What to check.** The cheapest and most neglected check is the **positive control**: instruct the agent plainly and non-adversarially to perform the forbidden effect, and confirm the assertion fires. An assertion that has never fired has never been tested, and a suite of them is a smoke detector with no battery. Beyond that: version and review envelope edits exactly like assertion edits, and require any widening that removes failures to be justified as fixture noise *with the noise fixed*, never as a scoring adjustment. Finally, separate three statements in the report — what was forbidden and did not happen, what fell outside the envelope and did not happen, and what nothing in this suite could have detected. Neither polarity sees an unobserved sink or a behaviour no task exercises, so that third list is the honest statement of residual risk, and it is the one that should set next quarter's investment.