In evaluating a model defense, what makes an attack adaptive rather than stock?

### The word does real work *Adaptive* here means adapted **to this defense**, not adapted during the run. It describes how the attack was authored: somebody read the description of the defended system and formulated an attack against that system. The contrast is a stock attack — an off-the-shelf search written for a plain classifier, which treats whatever you added as scenery. ### 1. The knowledge assumption The adaptive adversary is *granted* the defense's design, architecture and parameter settings by assumption. This is an evaluation convention, not a prediction about leaks: it is made so the resulting number cannot quietly include credit for the attacker not knowing what you built. The one thing not granted is genuine randomness drawn per inference — the adversary knows the distribution and the mechanism, not the individual draw. Candidates often get this backwards and argue that a realistic attacker would not know the design. Perhaps not today. But a claim resting on that ignorance decays with every document you publish, every binary you ship, and every hour somebody spends probing the deployed behaviour, and it is not a property of the model at all. ### 2. The objective is re-pointed This is the technically substantive half. A stock attack optimises against some quantity the undefended model exposes. A defense typically changes *which quantity a decision comes from*: | What the defense adds | What the deployed system now decides on | | --- | --- | | A preprocessing or transformation stage | the output after the transform, not before it | | A randomized component | behaviour in expectation over that randomness | | An ensemble with a vote rule | the aggregated verdict, not any single member | | A detector in front of the classifier | the joint outcome of accept-and-classify | An adaptive attack aims at the right-hand column; a stock attack keeps aiming at the left. That is the entire difference, and it explains the failure mode: the stock attack does not fail because the inputs are not there, it fails because it is optimising something the deployed system no longer decides on. ### 3. The awkward-stage argument Two design choices come up constantly as claimed defenses: make a stage non-differentiable, or make it random. **Non-differentiable.** Removing a usable derivative removes one *technique* for searching, not the inputs being searched for. The adaptive response is to work against a differentiable stand-in for the stage, or to use a search that needs only the returned decisions. So the defense buys the attacker a harder search, and it is the search — not the model — that the stock number was measuring. **Randomized.** Randomness makes each observation noisy, so more observations are needed and results vary between runs. It does not create a boundary, because the adversary can target what the system does on average over the randomness. The honest description of both is *this raises the attacker's cost by roughly this much*, which is a claim you can defend, rather than *this stops the attack*, which is a claim the first adaptive attempt refutes. ### 4. Effort, and its record The third component is unglamorous and is where most weak evaluations actually fail. An adaptive evaluation involves trying several formulations, discovering that some do not work, and continuing. A single defense-aware attempt that returned nothing is barely stronger evidence than a stock run. What raises confidence is a record: which formulations were tried, which were abandoned and why, and how much effort was spent before stopping. Effort spent is the unit of a negative result here, in a way it is not for a positive one. ### What adaptivity is not - **Not more steps.** Ten times the optimization effort on an objective that ignores your defense is ten times as much of the wrong search. - **Not more attacks.** A shelf of published attacks shares the assumption that broke all of them. - **Not an access class.** Adaptivity and access are independent axes. An attacker handed weights who runs an unaware attack is doing an unaware evaluation with excellent access; an attacker with only returned decisions who formulated the attack against your ensemble rule is being adaptive on a thin vantage. - **Not automatic.** No suite makes an evaluation adaptive on your behalf, because the adaptation is authorship. ### Why interviewers ask it Because it separates people who can recite the names of attacks from people who understand what an evaluation measures. The follow-up is usually the honest consequence: since adaptivity is authorship and authorship costs analyst-days, the strength of every non-break result you will ever read is capped by how many days somebody was willing to spend, and by what they could see while spending them.