A vendor calls their face-verification model "adversarially trained, so robust" — what does that leave an attacker free to do?

## The claim, and what is missing from it "Adversarially trained, so robust" states a training procedure and then asserts a property. The property does not exist on its own. Robustness is always **relative to a threat model**, and a usable threat model names at least four things: - **the perturbation family** — the geometry of changes the attacker is allowed to make (commonly stated as a norm); - **the radius** — how much change that family permits; - **the access assumed** — whether the attacker was given the weights and gradients, only returned scores, or only a top-1 decision; - **the attack actually run inside that budget**, and how hard it was run. Drop any of those and the number is not comparable to any other number, and not transferable to your deployment. ## Why the trained family is the boundary of the claim Adversarial training works by repeatedly asking, during training, "what is the worst input inside this allowed set, against the weights as they stand right now?" and then learning from it. The allowed set is a fixed geometric object chosen by the defender. What the model ends up resistant to is *that object*: it learns to keep its decision stable over the shapes of change it was repeatedly shown at their worst. Different families are different objects, and they are not nested in any convenient way: - a **small per-coordinate bound** lets every pixel move a little and none of them much; - a **few-coordinates budget** lets a handful of pixels move as far as they like and forbids the rest to move at all; - an **energy bound** limits the total size of the change but not how it is distributed. A point that is extreme in one of these can sit far outside the others. So resistance bought against one buys much less against another, and published evaluations repeatedly show cross-family robustness far below the headline figure — sometimes with a visible trade-off, where pushing hard on one family costs ground on another. ## The change that is not in any family The sharper gap is that some attacker moves are not magnitude-bounded at all. Turning the head, moving closer, changing the crop, or covering part of the face with an ordinary object are **geometric and area-bounded** changes. Measured in pixels, they are enormous — vastly larger than any radius a defense trains against — and yet a person looking at the result sees the same face. A defense priced in per-pixel magnitude does not resist these; it does not even have a unit in which to state them. This is why a robustness figure earned against a magnitude budget says essentially nothing about an attacker who is physically present in front of a camera and constrained instead by how far they can turn their head and how much of their face they may cover. ## Reading the number in the right direction A high robust-accuracy figure proves that **the attack that was run failed** at the budget it was run under. It does not prove that no attack succeeds. Two attackers with the same nominal "strength" but different families are simply different adversaries, and only the one that was tested was priced. So the first question to a robustness claim is not "how high is it" but "robust to what": which family, which radius, which access, which attack and how hard. The second is "what was *not* evaluated" — the families deliberately or accidentally left off the sheet are exactly where an attacker who has read the claim will go, because the claim tells them where the training effort was not spent. ## What to do with a vendor claim Treat the figure as characterising one adversary. If your deployment's realistic adversary is constrained differently — by physical access and a viewing angle rather than by a pixel budget — the vendor's number is true and irrelevant, and you need a measurement stated in the units your attacker actually works in.