The provider of a hosted model closes your submission as expected model behaviour rather than a vulnerability, but your client's application still exhibits the effect. What do you do with that item in the client's report?

## What the ruling is, mechanically "Expected behaviour" is a statement about the provider's bug bar, not about your client's risk. It means the observed output falls inside the envelope the provider considers the model's documented or intended behaviour, so no engineering work will be scheduled and the ticket closes. You cannot appeal it: you are not the provider's counterparty, your client may be. The finding's *truth* is untouched — the transcripts and the rate are exactly as valid the day after the ruling as the day before. What changed is the item's type: from "defect pending an upstream fix" to "a standing property of a dependency the client chose." ## Reframe the item Delete every phrase that implies something is coming. The heading becomes a property statement: under these conditions, at approximately this rate, this platform will do this. Everything below that sentence is now the client's decision. Then move the rating. "The model can be talked into describing X" is not rateable — there is no asset in the sentence. "This assistant, which is permitted to issue refunds, can be induced to issue one outside policy in roughly 12% of 200 crafted multi-turn attempts" is rateable, because severity lives in the capability the application granted, and that capability is precisely what the client can change. A provider verdict must never move the severity; severity follows impact on the client's system. ## Control classes and what each actually costs | control | cost | what it buys | |---|---|---| | remove the capability | the feature is gone; a product decision, not an engineering one | the only control whose effectiveness does not decay | | gate behind a human step | latency plus support headcount, measured in tickets per day times handling minutes | changes the outcome, but the business will push for a threshold that quietly restores most of the exposure | | constrain scope (tool, data, per-account limits) | roughly a day of engineering, no per-request cost | caps blast radius rather than frequency | | detect and block at the boundary | a second inference per request: a comparable-size judge model roughly doubles per-request cost and adds a few hundred milliseconds; a small fine-tuned classifier is single-digit milliseconds and cheap, with a worse error profile and its own threshold to tune | shifts a probability, and degrades against rephrasing | The first two change what a success is worth. The last two change how often a success happens. Say out loud which class you are offering, because a client who believes a filter closed the item will be unpleasantly surprised. ## Where the residual-risk number misleads Reports routinely say "the boundary filter reduces the observed rate from 24% to 3%", measured by replaying the same fixed suite through the filtered pipeline. Three things are wrong with reading that as a 3% residual. **Same-suite fitting.** The filter was tuned while looking at those prompts, so it has effectively been fitted to its own evaluation set. A held-out set of paraphrases written by someone who did not build the filter usually comes back materially higher. **Non-adaptivity.** The suite does not respond to the control. A person does: they observe what gets blocked and rewrite. A 24-to-3 drop against a frozen prompt list is not a claim about an adversary. **Denominator.** 3% of crafted attempts is not 3% of traffic, and a business that hears the latter has swapped the denominator in the direction that flatters the report. Write what the fraction is *of* into the same sentence as the number. One more measurement trap: a pre-control rate and a post-control rate taken weeks apart are not comparable if the endpoint drifted between them. Where you can, re-measure the unfiltered path in the same window as the filtered one and report both, so the delta is attributable to your control rather than to the provider quietly changing something. ## The ruling as procurement evidence, and the re-test Capture the ruling verbatim with its date. It converts a technical finding into a supplier input: if this is intended behaviour, then continuing on this dependency is a documented acceptance, and that is a conversation an owner can have with a vendor that you cannot have for them. Then record the item as an **accepted risk** with a named owner, the control, the residual risk, and a review date — not as closed, because closed means the risk is gone. Schedule a re-test of both the behaviour and the control. Intended behaviour today can drift tomorrow: your control may stop being necessary, or quietly stop being sufficient. A quarterly re-measure at 200 attempts with a judge is a few hundred calls and an afternoon, which is cheap next to discovering the drift from an incident. ## What to check Was the control re-measured on held-out prompts rather than the ones that shaped it? Is severity unchanged by the provider's verdict? Is the item recorded as accepted-with-an-owner rather than resolved? Is the ruling quoted with its date? And does the client understand which control class they bought?