The Cheapest Flip

## Two questions, not one measurement An adversary who can differentiate a model - because they pulled the weight file out of an appliance image, say, and run it offline - has two quite different things they can ask of a given input. **Fixed-radius (maximise damage inside a set budget).** Fix a norm and a radius first: every coordinate of the input may move by at most this much, or the total energy of the change is capped at this much. Then search inside that set for the change that hurts the model most. The result for one input is binary - flipped or not - and the result over a test set is a success rate, usually reported the other way round as accuracy under attack. **Minimum perturbation (minimise the change, subject to flipping).** Swap the roles. The flip is now the requirement, and the size of the change is the quantity being pushed down. The result for one input is not a yes or a no; it is a number - the smallest change the search managed to find that still made the model read the input differently. ## Why they are not interchangeable The common wrong answer in an interview is that these are the same experiment with the knob on a different side. They are not, and the giveaway is what each one returns. A fixed-radius run is one threshold cut through the distance distribution. It tells you what fraction of your data sits further from a boundary than the radius somebody picked - and it tells you nothing about how the rest of the distribution is shaped. Two models can report the same success rate at that radius while one has most of its data just barely on the safe side and the other has most of its data far away. The first collapses when the radius moves slightly; the second does not. A minimising run gives you the shape. You can ask where the mass sits, which classes or which slices of traffic sit closest, whether there is a thin tail of inputs that are practically on the boundary already. Crucially, it is what makes the fixed radius defensible: if you cannot say where your data actually sits, then the radius in your evaluation was chosen by convention or by whatever made the number look good. ## What each number is worth Both are empirical results produced by a search, so both are one-sided in the same direction, and this is the direction people get backwards: - A fixed-radius success rate is an **upper bound on robustness**. The attack that was run failed on those inputs. A better-resourced attacker lowers the number; nobody can raise it. - A minimum perturbation is an **upper bound on the true distance**. The search found *a* change of that size; a stronger search can only find a smaller one. It is never a guarantee that nothing closer exists - that direction requires a certificate, which is a different object entirely. So neither is a proof of safety. Both are records of what one adversary managed under one set of assumptions. ## What the attacker pays The formulations also cost differently, and that shapes where you see each one. A fixed-radius attack runs a bounded amount of work and stops; every candidate it produces is admissible by construction, because it stays inside the ball. A minimising attack has to hold two things in tension - keep the label flipped, make the change smaller - and it has to re-confirm the flip every time it shrinks. In practice that is one to two orders of magnitude more work per input. That cost is nearly free to an adversary running an extracted weight file on their own hardware, where the only bill is wall-clock time per item. It is close to prohibitive against an endpoint that meters and charges per call. This is why minimising results are typically reported by someone with the weights in hand, and why an adversary paying per query fills a budget instead. ## The payoff is different too There is also a difference in what the attacker wants. Filling a budget is about producing a flip at all. Minimising is about producing a flip **nobody notices** - an item that clears the model while the human who also looks at, or listens to, the same item registers nothing unusual. In a pipeline where a machine decision is spot-checked by a person, the smallest change is the one that survives the person, and the size of the change is measured against human perception rather than against a convenient constant.