How do you decide an LLM assistant is safe enough to launch, given ASR never reaches zero?

## The question has no lookup answer As of 2026 there is no accepted attack-success-rate threshold that means safe. Anyone who quotes one is quoting a number their own team made up. That makes launch sign-off a judgement call built from evidence, and the mark of seniority is constructing that judgement transparently rather than hiding behind a percentage. Start from the honest premise: attack-success rate is a function of attacker effort, and for anything with a public interface the effort available is unbounded. A frontier assistant reported around 0.1% success on a single attempt and roughly 5–6% after a hundred adaptive attempts; the shape, not the endpoint, is the fact. So the decision is never *is the rate zero* — it is *what happens the fraction of the time it is not, and can we live with that*. ## The three things sign-off actually rests on **1. Residual risk, measured and sliced.** Produce ASR against attacker budget, per severity category, judged by checks you trust. Cross-tenant data disclosure, unauthorised irreversible action and credential exposure get absolute counts; softer categories get rates. This is the only part of the package that is a measurement, and it is worthless without the budget attached. **2. Bounded worst case.** For every category where success remains possible, write the sentence that describes the realised harm. "An injected instruction can cause the assistant to summarise the current case file back to the user" is very different from "an injected instruction can cause the assistant to email a case file to an arbitrary address", even at identical ASR. The lever that changes the second sentence into the first is architectural — narrowing tool scope, removing the outbound channel, requiring approval for irreversible actions — and it is what makes a nonzero rate survivable. A launch package where the mitigation story is entirely "our classifier catches it" should not clear the bar, because detection-based defenses fall to adaptive attackers and are graded accordingly. **3. Detection and response.** Since you are shipping with a known residual rate, you are committing to catch and handle the cases it produces. That means trajectory-level logging you can actually reconstruct an incident from, a monitoring signal for each blocking category, a rollback or kill path for the feature, and a staged rollout so the first realised failure hits a small cohort. Shipping with 3% residual ASR and no detection is a worse position than shipping with 8% and a same-day response path. ## Spending a fixed pre-launch budget Given, say, three weeks before a public launch, the allocation that produces the most decision-relevant evidence: - **Adaptive attacks on the blocking categories first.** Depth on the two or three outcomes that would actually stop the launch beats breadth across twenty categories that would not. - **Autonomous auditing agents.** Tools such as Petri explore the system on their own and surface behaviours nobody wrote a probe for. Their unique value is coverage of the unanticipated — which, by construction, is where the failure that embarrasses you lives. Their output is also noisy, so budget triage time. - **Agent-level cases in a sandboxed copy of the real tool environment**, since a model-layer result does not describe your product. - **Least of all, growing the static list.** It is the cheapest activity and the one that most reliably produces a comforting number without new information. ## Contested ground — say so Several parts of this are genuinely unsettled and pretending otherwise is a tell. There is no consensus threshold. There is no agreed way to weigh a low-probability catastrophic outcome against a high-probability minor one for consumer products. Human approval as a control degrades through approval fatigue, so "a person reviews it" is a weaker mitigation at scale than it looks on a slide. And the regulatory floor is moving — transparency and general-purpose-model enforcement duties land in 2026 with higher-risk obligations phased later — so a package that clears an internal bar may still need a compliance answer. The right posture is to state the tradeoff shape and the assumptions rather than manufacture a consensus. ## What the sign-off should look like A one-page decision with: the categories that block and their measured counts at a named attacker budget; the worst realised harm per category with the containment that bounds it; the monitoring signal and response path for each; the rollout plan; the residual risk being accepted in plain language; and a named owner who accepted it, with a review date. It should be readable by someone who was not in the evaluation, and it should be the artefact you reread after the first incident — because the eventual postmortem's most useful question is whether the failure was outside the risk you accepted or inside it and simply undetected.