As a reviewer, what does the distribution of smallest perturbations an attacker needed per input tell you that one fixed-radius success rate does not?

## What one number leaves out An evasion evaluation reported as a single figure - accuracy under attack at one perturbation radius - is a threshold cut. Somewhere behind it is a distribution: for each input, the smallest change an attacker had to make before the model read it differently. The reported figure is just the fraction of that distribution lying above the chosen radius. Everything the cut discards is the interesting part. **Shape near the threshold.** If the mass of the distribution is bunched just above the reported radius, the number is standing on a cliff: nudge the radius and it collapses. If the mass sits far above, the number is on a plateau and is genuinely descriptive. From the single figure you cannot tell these apart, and they mean opposite things about the deployment. **Where the exposure is concentrated.** Aggregate rates hide slices. It is common for one class, one channel, or one population of inputs to sit systematically closer to a boundary than the rest, so an overall figure that looks acceptable is carried by the easy majority while the part of the traffic that matters is the part sitting close. **The tail.** A handful of inputs practically on a boundary is a different operational problem from broad mild fragility, and needs a different answer. The distribution shows the tail; the rate does not. **Whether the radius was chosen honestly.** This is the reviewer's real question. A radius fixed by convention before any results existed is one thing. A radius arrived at after seeing which value made the model look best is another, and a reader cannot distinguish them from the number alone. Publishing the distances is what makes the choice auditable, because the reader can see for themselves whether that radius was a fair place to cut. ## Two things the distribution still is not **It is an upper bound, everywhere.** Each per-input distance is the smallest change *some search found*, not the smallest that exists. A better-resourced adversary, a different formulation or more restarts can only move a given input's number down. So the whole distribution shifts left under a stronger attacker and never right, and no part of it is a guarantee. A statement in the other direction - that nothing smaller than some radius can flip this input - is a certificate, a different object with its own confidence and its own cost, and it is not what an attack produces. **It is measured in the model's input units.** A distance is expressed in whatever space the model consumes, so two models with different scaling, normalisation or resampling produce numbers that cannot be put side by side. Comparing them without accounting for that is a category error, and it is a common one in vendor material and internal decks alike. ## Using it as a reviewer When a robustness result lands on your desk, the productive questions are all about what the single number hides: - Which radius, in which norm, and was it fixed before the run or after it? - Is there a distance distribution behind it, even on a sample of a few hundred inputs? - Does the reported figure hold up when the radius moves a little in either direction? - Is the exposure even across the classes and traffic slices you actually care about? - Was the search that produced these numbers the strongest one available, or the most convenient one? A team that can answer these has measured something. A team that can only cite one figure has run one experiment and reported one cut through it. ## What it changes about a defence claim The distribution is also the cleanest way to see whether a hardening effort did anything real. A defence that moves the whole distribution to the right has genuinely bought distance. A defence that moves only the inputs that were sitting immediately below the evaluation radius, leaving the rest of the shape unchanged, has bought a headline number and very little else - and that pattern is invisible if only the rate is reported. The distance distribution turns a claim of the form 'we improved robustness' into something a reviewer can check.