A turnstile face matcher reports 91% robust accuracy at a per-pixel radius — what does that say about an attacker limited to head pose and occluded area?

## Two adversaries, one number The vendor's figure describes an adversary who can edit the input as data, constrained by how far each pixel may move. Your deployment's adversary stands in front of a camera. They cannot touch the file at all. Their constraints are **geometric and area-based**: a range of head pose and viewing angle they can hold naturally at a gate, and a fraction of the face they can plausibly have covered without drawing attention. Those two budgets are not a strong and a weak version of one thing. They are stated in different units. A change of viewing angle moves essentially every pixel by a large amount, so it is nowhere inside a small per-pixel radius; conversely, nothing inside that radius corresponds to turning one's head. The correct description of the vendor's coverage over this attacker is **not measured** — not "weakly robust." ## What the attacker's vantage adds A realistic attacker here also has more than physical presence. A previously released on-device weight file for an earlier generation of the same matcher is a **differentiable stand-in**: they can compute against it freely, with no queries to your gate and no log entries for you to notice. It does not need to match your deployed model's accuracy, and it does not need the same architecture. It only needs to agree with the deployed matcher **near the boundary being attacked** — the region separating "this is the enrolled badge-holder" from "this is not." That is a much weaker requirement than functional equivalence, and it is why transfer works at all. Combine the two and the picture is: offline optimisation against a stand-in, delivery through pose and occlusion, against a defense that priced neither. ## The payoff is not the metric the vendor reported Robust accuracy is an average over a test set. The attacker at your turnstile does not care about the average. They want **one targeted acceptance**: one chosen identity admitted, once. Two consequences: 1. **Averages hide it.** A matcher can hold its aggregate numbers exactly while being reliably wrong on one carefully chosen pair. The reported figure would not move. 2. **The relevant rate is per-identity and targeted**, not aggregate. Some enrolled identities are much easier to reach than others, and the attacker chooses which one to be. So even a robustness number measured in the right units, but reported as an average, would answer a question you did not ask. ## What you should ask for instead Stated in the units of the adversary you actually face: - targeted acceptance rate for a chosen enrolled identity, **across the pose range** the gate accepts and **at the occlusion fractions** it tolerates; - measured on the **deployed capture path** — the same camera geometry, lighting envelope and enrolment pipeline — because a result obtained on clean stored imagery is a different experiment; - with the attacker's assumed vantage stated: whether they were allowed a stand-in model, and whether they were allowed to observe accept/reject outcomes at the gate. And from the vendor, the honest scoping line: which families were evaluated, at which radii, and which were not evaluated at all. ## Triaging a finding here A related trap: an attempt that succeeds once in five tries is still a finding. A gate that admits the wrong person on the fifth attempt is not "mostly holding" — the attacker can retry, and retries at a turnstile are cheap and unremarkable. Report success rate over attempts under the stated budget, not a binary. ## The one-line version The 91% is a true statement about a magnitude-bounded attacker on a test set. Your attacker is bounded by degrees and by area, and wants one specific acceptance. Nothing in the reported figure covers that, and nothing about it is dishonest — it simply answers a different question.