In a character-chat product, a persona scene returns fluent in-character content on a refused topic. What can the tester claim?

## Separate the claims before you write the finding A scene that returns refused-category material supports a stack of claims that people routinely collapse into one: 1. The model generated text in a category it normally declines. 2. The persona framing is what moved it there. 3. The text is accurate. 4. The text gives a reader capability they did not already have. Claims 1 and 2 are what you observed, and 2 needs at least a comparison against the same request asked plainly. Claims 3 and 4 are the ones that decide severity, and the scene tells you nothing about either. ## Why the frame is the worst instrument for accuracy In-character generation optimises for consistency with the scene. The character has a voice, a stake and a next line, and the strongest constraint on that line is that it fits. Where the model holds real material, some of it may surface. Where it holds none, the scene still needs continuing, and a hedge or an admission of ignorance breaks the fiction the user built. So the model supplies something plausible in the same confident register it uses for the parts it does know. That is not an occasional failure mode of the family; it is the family's normal output. And crucially the two cases are indistinguishable at the point of reading: both arrive fluent, both arrive in voice, and neither carries a marker. You cannot separate them from inside the product, no matter how many turns you spend. ## What this does to a report A finding that says *the model produced instructions for X* is a different assertion from *the model produced correct instructions for X*, and only the second supports an uplift argument. The honest shape is to state what was observed, state that accuracy was not assessed, and if you do assess it, say what you compared against. Verification means checking the content against ground truth in the relevant domain, outside the model, by someone competent to judge it. In most of the categories where this matters, that is expensive, sometimes legally awkward, and often outside the tester's own expertise. That cost asymmetry is the whole practical story of this family. Producing fluent refused-category prose in a scene is cheap. Establishing that any of it is true is expensive. An attacker faces exactly the same asymmetry, which is why the family's payoff is weaker than its reputation: they have paid for something they cannot trust without leaving the product. ## Two adjacent errors The first is treating one successful scene as a rate. Generation is probabilistic and the surrounding turns differ, so a single success establishes that the construction worked once against this deployment. Whether it is a finding or a curiosity depends on how it behaves across runs, and reporting a single run as if it were reliable is how a triage queue fills with things nobody can reproduce. The second is inferring the failing component. Refused-category output arriving in full does not tell you whether there was no output screening, whether a screen scored it below its threshold, or whether the response was streamed and the remainder was cut after tokens had already been sent. Those are different observations with different shapes, and the report should say which one you actually saw. ## Severity, honestly A reasonable severity argument for this family usually rests on claim 1 alone: a product that will produce this category of material in voice has a control gap, and that is worth filing on its own terms, especially in a consumer product where the material reaches a user directly. What it should not rest on is an unverified assertion that the content is real and dangerous. Interviewers ask this question because inflating that step is the single most common flaw in AI red-team reporting, and because a candidate who volunteers the confabulation problem before being pushed on it has clearly done the work rather than read about it. ## The one-line answer The scene established a refusal-behaviour failure, not a disclosure. Fluency in voice is evidence about the model's willingness to continue, and about nothing else.