How can an input built against the attacker's own classifier fool one they have never queried?

### The situation The attacker wants a specific input to be read wrongly by a classifier they do not own. They have no account on it, no query access, no way to see its scores or even its top-1 label. What they do have is a model of their own that does the same job — one they trained, or downloaded, or fine-tuned themselves. They craft the input against **their** model and submit it once to the target. Often it works. That property has a name: *transferability*. ### What a crafted input actually is Start with the object, because the most common wrong picture of it makes transfer look impossible. An adversarial input is not a noisy input. The attacker computes the gradient of a loss with respect to the **input** — not with respect to the weights, which is the ordinary training operation pointed a different way — and that gradient names the direction in which a small change most increases the model's error. They move the input a small distance along that direction, inside whatever budget they have set themselves. Random noise of the same size essentially never flips a trained classifier. The direction is the entire attack; the magnitude is only how far they were willing to go. ### Why a second model agrees Ask where the direction comes from. It comes from the features the source model learned to key on when it separates the classes. Two models trained for the same task, on data drawn from the same kind of distribution, learn overlapping features — including the brittle, non-robust correlations that happen to predict the label on that distribution but that no person would call the reason for the answer. Both models therefore place a boundary in roughly the same region of input space, because both are separating the same classes over the same material. A direction that carries an input across one of those boundaries points broadly the right way for the other. Notice what this argument does **not** require. It does not require the same architecture, the same layer widths, the same optimiser, the same training recipe or the same random seed. It does not require the attacker to have seen a single one of the target's own training rows. It requires a similar task and an overlapping data distribution — and a model built to solve the same problem over the same kind of inputs has both by construction. ### What the attacker buys The payoff of the zero-access version is not just the misclassification, it is the absence of everything else. No query budget is spent and no rate limit is touched. There is no sequence of near-miss probes for anyone to find in a log afterwards. From the target's side, one ordinary-looking submission was scored the wrong way, and there is nothing in the traffic that separates it from an honest one; the only signal available is the outcome, not the request pattern. ### What it costs Transfer is bought at a real price, and a candidate who describes it as free has missed the mechanism. What generalises between the two models is the **direction** the loss suggested, not the exact point the attacker landed on. A perturbed input typically sits only just past the source model's boundary; the target's boundary lies nearby but not identically, so a good fraction of crafted inputs land back on the correct side of it. Success rates for transferred attacks run well below the same attack executed with the target's own weights in hand. The gap widens when the attacker wants a *specific* wrong answer rather than any wrong answer, and it widens as the two models' training data and architecture families diverge. ### Where it stops Transfer degrades when the two models are genuinely solving different problems over different distributions — not when one of them is merely secret. It degrades sharply for targeted goals. It does not survive an assumption change: an input crafted digitally and then printed, photographed or re-encoded is a different problem, because a capture pipeline destroys small perturbations regardless of which model they were built against. And on discrete inputs such as text or binaries there is no small continuous step to take; the available moves are behaviour-preserving edits, and results measured on continuous inputs do not carry over unchanged. ### How to say it in an interview One sentence for the mechanism — models trained on the same kind of data for the same task agree about where the boundary goes, so the direction that crosses one crosses the other — and one sentence for the price — the direction transfers, the exact point does not, which is why transferred attacks succeed far less often than white-box ones.