Your harness substitutes adversarial content into an agent's tool results and the agent never misbehaves. Before reporting the agent as resistant, how do you verify the content actually reached the model?

Negative results from an interception harness are the ones most often wrong, because the failure mode is silent. You handed a shim some text, the shim returned it, the run finished clean. Nothing in that chain establishes that the model ever read a single word of it, and no component raised an error to tell you otherwise. ### Where content disappears between the shim and the context window The gap you have to close is between *the value your shim returned* and *the rendered context* -- the exact message list sent to the model on the turn after the intercepted call. Several ordinary scaffold behaviours destroy a payload in that gap: - **Length truncation.** Most agent scaffolds cap tool-result length, often at a few thousand characters. A payload placed late in a long result is simply gone, with no error. - **Summarisation.** Some scaffolds compress tool output before insertion, often with a cheaper model. An instruction survives as a paraphrase, or not at all. - **Serialisation and escaping.** A result forced into a schema may be re-encoded; newlines, markup or delimiters that carried the payload's structure are neutralised. - **Validation and error substitution.** An unexpected shape can be replaced wholesale by a generic error string, and that string is what the model then reads. - **Caching and dedup.** A repeated identical call may be served from a result cache, so your later substitution never reaches the model. - **Wrong intercept point.** The agent used a different tool, or a different code path, than the one you wrapped -- and a harness that logs "substituted" for a call the agent never made looks identical to one that worked. ### The verification loop Emit a **canary**: a distinctive, harmless marker string inside every substituted result. Then assert three things in order. (1) The canary appears in the rendered messages for the next model turn. (2) The payload text appears alongside it, unmodified. (3) For the strongest check, run a probe turn asking the model to quote back what the tool returned -- this catches the case where the head of a long result survived truncation and the tail, carrying your instruction, did not. Also record *where* the delivered text landed: a payload sitting at the very end of a long context is a materially weaker test than the same text delivered where the workflow would naturally have returned it. ### What verification costs The canary itself is free -- a dozen tokens per result and a substring assertion. Capturing rendered context costs storage and a redaction pass, because those transcripts contain the full tool payloads. The quote-back probe is the real line item: it adds one extra model call per case, roughly a 10-15% inference increase on an eight-turn task, and it perturbs the run, so it belongs in a separate verification pass rather than inside the scored run. Wiring the context dump into a framework that does not expose the final message list is often half a day, and re-running the cases that fail delivery costs a full extra pass. Against that, the cost of skipping it is a resistance claim the defending team uses to close remediation work. ### Where the number misleads "0% attack success at the tool boundary" is the most dangerous number a harness can emit, because it is consumed as evidence of a defensive property. If the true cause is that the scaffold truncates every tool result at 2,000 characters, the honest finding is different and more useful: an *incidental* mitigation, unowned and undocumented, that disappears the day someone raises the cap or adds a long-context tool. The second misreading is aggregate: mixing delivery-verified nulls with unverified ones inflates the apparent coverage of the run. Ten verified nulls and ninety cases whose payload never arrived is not a 100-case resistance result; it is a 10-case result and a delivery bug. Report the two denominators separately -- cases attempted, and cases where delivery was confirmed. ### What you would check Before recording any null, confirm the canary in the rendered context, confirm the payload's tail with a quote-back probe, and confirm the intercepted call is the one the agent actually made. Treat an unverified null as an **invalid run**, not a negative one: fix delivery and re-run before it touches a coverage table. When delivery consistently fails for a structural reason, stop and write that up on its own -- the truncation limit, the summariser, the cache -- because that is a real property of the target that nobody on the owning team has necessarily noticed.