With unlimited free re-scans of a local malware classifier, why does forcing one chosen verdict still cost more attempts?
answer
- count the outcomes that end the search
- many acceptable outcomes against exactly one
- a bare verdict says only not yet
- price depends on which class you name
basics
~10 sThe stopping rule is narrower. An any-wrong-verdict run ends at the first outcome that is not correct; a chosen-verdict run must pass those and keep going, burning more attempts and more restarts.
solid answer
~40 sFree re-scans remove the money cost, not the attempt cost, and the two goals consume attempts at different rates. An untargeted run has a disjunctive stopping rule: any of the wrong outcomes ends it. A targeted run has a single condition, so every wrong-but-unwanted verdict it encounters is a failed attempt, and it restarts or keeps searching. With a verdict-only oracle the gap widens, because the returned label gives no partial credit toward the class the adversary named, so progress is measured only by whether the run has already finished. The targeted cost also depends on which class is named: a verdict the model already considers close is far cheaper to reach than a distant one, which is why a single targeted rate with no class attached tells you very little.
go deeper
Know that a chosen-verdict attack is the harder of the two and that the difference is counted in attempts per file, not in how clever the attacker is.
Explain it as a stopping rule: one goal ends at any wrong outcome, the other only at a named one, so the second spends attempts on outcomes it cannot use. Add that the price depends on which class is named.
Be able to say how the endpoint's output format reprices each goal, and why an attempt budget must be stated for a result against a classifier the adversary holds locally, where nothing meters them.
Be ready to argue what your product should return to callers, knowing that withholding scores raises the cost of chosen-verdict attacks without removing verdict-only ones, and that the cost lands on legitimate consumers too.
## What the attempt budget actually buys Set the scene precisely, because it is where the intuition usually goes wrong. The adversary holds a copy of a malware classifier that runs locally and returns a verdict, benign or malicious, for any file handed to it. Nobody meters that. There is no bill, no rate limit and no log the defender sees. The only budget left is *attempts per file*: how many modified variants of one sample the adversary is willing to produce and re-score, plus how many times they abandon a line of attack and restart. Under that budget the two goals are not equally priced, and it is worth being exact about why, because "targeted is harder" is a claim a middle-level interview expects you to unpack. ## Two different stopping rules An untargeted run stops the first time the verdict is anything other than the correct one. On a k-class model that is a disjunction over k-1 outcomes; on a two-class scanner it is the single other class, but the run is still stopping at the first crossing out of the current region. A targeted run stops only on one named class. Outcomes that would have ended an untargeted run are, for a targeted run, failures that consumed an attempt. That is the whole mechanism. It is not that the targeted adversary is doing something more sophisticated; they are doing the same work against a strictly narrower acceptance condition, so at any fixed number of attempts fewer runs have finished. Two consequences follow: - **The gap grows with the number of classes**, since the untargeted rule accepts more outcomes while the targeted rule still accepts one. - **The gap grows with distance to the named class.** Forcing a verdict the model already ranks near the top of its ordering is far cheaper than forcing one it ranks nowhere. So the targeted cost is really a cost *per target class*, and reporting one aggregate number hides an enormous spread. ## Why a verdict-only oracle hurts the targeted goal more What the oracle returns changes the price. When a scanner returns only a label, the adversary learns one bit per attempt: finished or not finished. An untargeted run can live with that, because the bit it needs is exactly the bit it gets. A targeted run wants to know whether it is getting *closer to the named class*, and a bare verdict never says that; it says only "not yet". Progress therefore comes from restarts and from carrying over what worked on similar files, rather than from feedback within a run. Give the adversary a score or a full distribution and the targeted goal gets dramatically cheaper, because the returned numbers rank the classes and the adversary can see the named one rising. That is the honest reason a defender thinks about what an endpoint returns. It is a cost control on one family of attack, never a boundary: hiding scores raises the bill and removes score-driven search, while an attack that needs nothing but the returned label still works. ## Free does not mean cheap, and cheap does not mean unbounded Because the scanner runs offline, the attempt budget is bounded only by the adversary's patience and compute. A defender's instinct here is to reach for something that caps attempts, and that instinct is void: metering assumes a served endpoint, and this adversary has the model on their own machine. So the correct way to report a result against a locally held classifier is not "it succeeded", but "it succeeded at roughly N attempts per file", with N stated, precisely because N is the only thing left that separates the two goals. ## The ordering flips before training Everything above is about an adversary acting at inference on a finished model. Move them to the other side of training, where they can get rows into the corpus the next model is fitted on, and the ordering inverts. An adversary who wants one specific behaviour on inputs they choose is writing that association directly into the fit, and does not have to search for it afterwards; an adversary who wants the model broadly worse is fighting the entire clean majority of the corpus. Aiming becomes the cheap goal and degradation the expensive one. How many inserted rows each of those actually costs is a separate question with its own answer; what belongs here is only that the ordering is a property of *when* the adversary acts, not of the goal itself. ## What to take into an interview State the mechanism as a stopping rule, not as a feeling. Note that the targeted price varies by named class and is meaningless as one number. Note that the oracle's output format moves the price of the targeted goal much more than the untargeted one. And be ready for the follow-up that moves the adversary to training time, where the same axis carries the opposite price tag.
- Does the same cost ordering hold if the adversary can write into the training corpus instead?No, it inverts. An adversary aiming at one specific behaviour writes that association into the fit directly and never has to search for it at inference, while making the model broadly worse means fighting the whole clean majority of the corpus. So aiming is the cheap goal before training and the expensive one after it. The actual insertion counts each goal needs are a separate question.
- If the scanner returned a confidence score instead of a bare verdict, which goal benefits more?The targeted one, by a wide margin. A score ranks the classes, so the adversary can see whether the class they named is rising and get feedback inside a run rather than only at the end. An untargeted run already receives the bit it needs from a bare verdict, so it gains comparatively little.
- Why is one aggregate targeted success rate on a multi-class model a weak statistic?Because the cost of forcing a class varies enormously with how the model already ranks it. Near-neighbour classes fall quickly, distant ones may not fall at all inside the budget, and averaging over both produces a number that describes no particular adversary. The useful form is per target class, with the attempt budget stated.
saying these in an interview costs you the question
- Says targeted is harder because it needs parameter access
- Assumes the attempt cost is the same for every target class
- Thinks a free offline oracle makes both goals equally cheap
- Believes hiding scores removes chosen-verdict attacks rather than repricing them
- Carries the inference-time cost ordering back to training-time attacks