Why is a white-box attack result treated as a ceiling and a query-only result as a floor?

## The classes are nested, and that is the whole argument Access classes in adversarial ML form a containment order. An adversary assumed to hold weights, architecture and input gradients can do anything an adversary holding only returned predictions can do — it can send queries too — plus more. There is no capability the weaker one has and the stronger one lacks. From that single fact the ceiling-and-floor reading follows directly, with no experiment required: - the **white-box attack success rate** is an upper bound on the success rate of any weaker adversary against the same model, under the same perturbation family and radius; - the **accuracy surviving** that white-box attack is correspondingly a lower bound on the accuracy surviving weaker adversaries; - a **query-only attack success rate** is a lower bound on what a query-only adversary can eventually do, because it reflects one budget and one method. ## Why the two numbers are different kinds of claim They are not two estimates of one quantity. The white-box figure answers 'how bad can this get, at most, within the assumed perturbation family'. The query-only figure answers 'how bad did it get for this attacker, with this budget, using this technique'. The first is stable in the sense that a *weaker* adversary cannot exceed it. The second is unstable upward by construction: give the same attacker more queries, a better search, or a stand-in model that agrees with the target near the boundary being attacked, and the number rises. Nothing about a query-only result caps anything. This is why quoting a query-only number alone understates risk, and why an evaluation that reports only 'our model resisted an external attacker' is a weak claim. It is also why quoting only a white-box number can mislead in the other direction: it may describe an adversary far stronger than the one your deployment actually faces, and treating it as the expected case can push spending toward the wrong control. The honest report carries both, each with its access class named: | Reported figure | What it bounds | | --- | --- | | Accuracy under the granted-access attack | The floor for every weaker adversary | | Accuracy under a query-only attack | Nothing; it moves down with budget and technique | | A figure with no access class stated | Nothing at all | ## What the gap between them is worth The distance between the ceiling and the floor is the value of not handing out weights, and it is real. It shows up as the adversary's bill: queries that are metered or paid for, wall-clock spent probing, or the effort of fitting a stand-in that need only agree with the target near the boundary under attack. That last point is why the gap is narrower than people expect — the substitute does not need the target's accuracy or its architecture, only local agreement where the attack is working. What the gap is *not* is a boundary. It is a cost control. Costs are paid by adversaries who care, and they fall over time as techniques improve, so a defence whose entire value lives in that gap is a defence with an expiry date and no floor underneath it. Reducing what a prediction reveals shifts an adversary between families and raises the bill; it does not remove the possibility of attack, and the ceiling is unchanged by it. ## The direction discipline Getting the direction of these claims right is most of the skill. A white-box run that leaves high accuracy means *the attacks that were run failed*; it does not certify anything, and a stronger search later can only lower it. A query-only run that leaves high accuracy means even less. And the ceiling property holds only when the two runs share everything else — same model version, same perturbation family, same radius, same task. Two rows compared across different radii or different norms are not comparable at all, and the containment argument does not apply between them. ## Using it in practice When you read a robustness claim, the first question is not 'what is the number' but 'what access did the attacker have'. Then: is this the ceiling or the floor? If it is the floor, the model may be far worse than stated. If it is the ceiling, the number is the one you can act on — it is the worst case you have evidence for, and everything you actually face sits under it.