Your two-step mail-exfil payload worked once then failed four tries — is that a finding?

## The question behind the question An interviewer who asks this is testing whether you can characterise a **probabilistic finding** honestly — neither inflating one success into a reliable exploit nor discarding a real pathway because it mostly failed. On this class of attack that judgement is the whole job, because the method is chancy by construction. ## Why this method is probabilistic by construction Several independent sources of variance stack up: - **One obeyed instruction.** The payload is a single injected instruction; there is no interactive loop to adjust. - **One shot per victim.** Realistically you get one meaningful attempt before the mail is read, deleted, or the behaviour is noticed. You cannot grind. - **Blind targeting.** The target is named without visibility; it may not exist in this mailbox at all, so some attempts fetch nothing for reasons unrelated to the payload's quality. - **A truncatable collect step.** A per-turn tool-call budget or an approval deadline can end the run before the fetch completes, independent of anything else. Given all that, **one success in five attempts is an expected shape**, not an anomaly. A deterministic bug either reproduces or it does not; this does not behave that way, and treating it as if it should is the mistake. ## The two failure modes of the triager, and of you There are two ways to report this badly: 1. **Overclaim.** Rounding a single win up to 'a reliable exfiltration exploit' is dishonest and erodes trust; the next reviewer who cannot reproduce it discounts everything you file. The method is opportunistic and the report should say so. 2. **Underclaim / dismiss.** Throwing the finding out because it failed four times hides a real pathway. A single unflagged success can still remove real data from a real mailbox. Variance is not the same as absence of risk. ## How to report it honestly Separate the **mechanism** from the **hit rate**: - **Mechanism / pathway.** State plainly what was demonstrated: an injected instruction caused the assistant to fetch content the user never surfaced and emit it. That pathway either exists or it does not, and here it does. - **Firing conditions.** Name what has to be true for it to succeed: the target present, the descriptor matching, the fetch completing inside the budget and before approval. - **Reliability with its denominator.** Give '1 of 5', not a bare '20%' floating free — the denominator and the conditions let a reader weigh it. A fraction with no context misleads in either direction. The triager should come away understanding that the *risk is real* and the *exploitation is opportunistic*: not something an attacker fires with confidence, but something that, when it lands unnoticed, is a genuine loss. ## Why you cannot just rerun for confidence With ordinary code you raise confidence by re-running until the behaviour is stable. Here each run is a fresh blind attempt against a possibly-different mailbox state, and re-running does not converge the way a deterministic exploit does. You also may not *get* many honest reruns per victim. So confidence comes from understanding the conditions that make it fire — the mechanism — not from a pass/fail count you would trust for deterministic software. ## What a good answer sounds like A strong candidate says 'yes, it is a finding' immediately, then explains *why the low rate is expected* and *how to report it without over- or under-claiming*: mechanism, conditions, reliability with a denominator. A weaker candidate either dismisses it ('it only worked once, not real') or overstates it ('it works, ship the exploit'), both of which misread a probabilistic finding.