Instructions in a Picture

## The two claims a pasted image makes, and the one it does not Picture an assistant that holds one person's mailbox, calendar and contacts. The user is sent something, screenshots it, pastes the screenshot into the conversation and types "deal with this". The application now has one turn with two parts: a short instruction the user wrote, and an image the user did not write. Almost everything that goes wrong here follows from collapsing those two into "the user's message". ### Trust attaches to the sender, not the author The turn arrives from an authenticated session and the application's message structure marks it as coming from the user. That structure records which participant transmitted the content. It makes no claim whatsoever about who authored the bytes inside it. Forwarding is the everyday version of this: a colleague you trust can hand you a document written by anyone at all, and your trust in the colleague says nothing about the document. A paste is a delivery act. ### A multimodal model reads the frame, not the point Given a picture, the model produces a reading of what is legible in it: the message the user cared about, the interface chrome around it, a footer strip, a caption, a quoted tail, anything resolvable at the fidelity the pipeline handed over. There is no marker inside an image saying "this region is the part the user meant". A sentence written in the imperative and addressed to an assistant therefore sits in the model's context as readable content, alongside the application's own instructions, and the model's preference for its own instructions is a trained tendency rather than an enforced boundary. ### This channel carries less provenance than the ones people worry about Compare how untrusted text usually reaches a model: | how the text arrived | what travels with it | | --- | --- | | a retrieved chunk | a source identifier and index metadata | | a fetched page | a URL, and a fetch some code initiated | | an uploaded file | an upload event, and whatever runs on uploads | | a pasted screenshot | only the fact that this account pasted it | Nothing fetched the screenshot, nothing indexed it, and a screen that matches on characters sees an image rather than a sentence. What stands between the picture and the model is the user's own glance before they pasted, and looking at an image is not the same act as reading the text in it: people paste an image for one thing in it and their attention stays on that thing. ### Why this is an injection question and not a content question Injection aims at the application's own instructions using data the application feeds the model. Jailbreaking aims at the model's trained refusal. The picture in this scenario does not need the model to say anything it would otherwise refuse; it needs the assistant to do something different with a mailbox it already has permission to use. By channel the span arrives inside the user's own turn, which superficially looks direct; by authorship it is second-order, because the span was written by whoever built the image and the trusted user is the delivery mechanism. In a report, what matters is the target and the authorship, not which turn carried it. ### What obedience proves, and what it does not If the assistant follows the planted sentence, that proves the span reached the context and was read as directive. It does not prove that any stage was bypassed, that no screening ran, or that the application's instructions were removed. Conversely, if the model declines, that is the answering model refusing to produce something; it is not evidence that a screening layer blocked the image. A refusal and a block are different events with different shapes, and telling them apart is a measurement rather than an assumption. ### Where the construction stops working The crop the user takes may exclude the region entirely. The user's reason for pasting may be the image's text itself, in which case they read it. The deployment may never pass the picture to the model. The payoff may need a capability the assistant does not hold. Each is a separate failure mode with its own rate, which is why a single successful paste is one observation against one configuration rather than a reliable finding. ### What a finding should actually say Name what carried the span (a picture inside the user's own turn), what it was aimed at (the application's instructions), what was in the way (only the pasting user's attention), and what was observed, how many times, against which configuration. Anything stronger overstates a probabilistic result.