As the lead of an engagement you receive one artefact: a nonsensical suffix, found by a token search on weights your organisation hosts, that reliably drives that model to produce disallowed content. Which defensive decisions does that result legitimately support, and which does it not?

**Separate three claims that routinely get merged.** The artefact on the table is a suffix, found by gradient-guided token search against weights the organisation hosts, that reliably drives that checkpoint to emit disallowed content. It licenses claim (1) and neither of the others. 1. *This checkpoint will produce this content given a sufficiently strange input.* Proven, on those weights, at the decoding settings tested. This is a real and durable finding about the model. 2. *A user of our product can obtain this content.* Not proven. The deployed path has its own input surface (does it even accept arbitrary strings, or is the free-text field templated?), possibly an anomaly or fluency check in front, its own system prompt, its own decoding settings, and output-side controls. Every one of those can break the reproduction. 3. *Models in general, or some vendor's hosted model, behave this way.* Nothing here supports it. The search required weights the team holds; cross-model behaviour is a separate measurement with separate access, separate cost and a separate scope. **Decisions the artefact legitimately funds.** - **Layered controls over another round of refusal tuning.** The result shows refusal behaviour is shallow off-distribution — it holds on inputs that resemble training data and gives way on inputs that do not. More refusal fine-tuning is optimising the same shallow surface; input anomaly checking and output-side checking are different layers. - **Coverage of every path into the context window.** Anomaly checking on the user turn is the easy half. Retrieved documents, tool and API output, uploaded file contents and prior conversation state reach the same context and frequently bypass input-side scoring entirely. - **Output-side checking specifically.** It is indifferent to how strange the input that produced the content was, which makes it the layer that does not degrade as attacker technique changes shape. - **A checkpoint-pinned regression case.** Keep the suffix, the target, the decoding settings and the expected verdict, and re-run at every fine-tune, merge, quantisation change or release — because the artefact is bound to the weights it was optimised against and will silently stop reproducing. **Decisions it does not support.** A customer-facing severity rating, or any public exploitability claim, before someone walks the real deployed surface. A comparison or ranking of models the team holds no weights for. And — the inverse error — removing or declining to fund a control on the grounds that "the filter caught it anyway", which promotes a single form-based check into a sole control with a tuned threshold and a known false-positive bill. **What it costs, and why the cost belongs in the report.** The attacker side: local weights, GPU-hours per behaviour per checkpoint, and a full re-run every time the checkpoint moves. The defender side against this artefact shape: one forward pass of a small scoring model per request. Put both numbers in front of leadership and the escalation conversation stops being an argument about adjectives — "critical" versus "informational" — and becomes a comparison of prices for a class of attack. That reframing is most of the value a principal adds to a write-up like this. **Where the number misleads.** Two directions, and the second is more common than people expect. *Over-claiming*: a weights-level reproduction rate quoted as a product exploitability rate, when the deployed path was never exercised — the denominator quietly changed from "attempts against raw weights" to "attempts against the product". *Under-claiming*: the team looks at the gibberish suffix, concludes a fluency check would reject it, and quietly drops the finding. That loses the durable half — the model itself was willing — and it is the failure mode that costs organisations most, because nothing gets written down and the next checkpoint is never re-tested. **What you check before signing the report.** That the checkpoint hash, decoding settings, target string, matcher and budget are all recorded. That someone has attempted the reproduction through the real surface, and that the outcome of that attempt — success *or* rejection at the door — is stated explicitly rather than left blank. That the regression case is attached to a checkpoint and has an owner. And that the write-up names, in one sentence, what a defender pays to catch this input shape against what the attacker paid to build it.