Why is an adversarial perturbation budget's norm and radius a threat model rather than a tuning knob?
answer
- an unconstrained attacker is not attacking
- two halves, both load-bearing
- who can touch what, and how much
- everything a little versus a few a lot
- change the pair, change the attacker
basics
~20 sThe norm and radius together describe an attacker: a per-coordinate cap lets every input value move a little, a sparse budget lets a few move a lot. Change the pair and you have evaluated a different adversary.
solid answer
~50 sAn evasion result is only meaningful relative to what the attacker was allowed to change, because an attacker with no constraint can simply submit a different input and the model's new answer is correct rather than fooled. So the change is bounded by a norm, which says how size is measured, and a radius, which says how much is allowed. That pair is the attacker description, not an experiment setting. A per-coordinate cap (an L-infinity ball) grants broad, shallow write access -- everything moves a little, like a calibration drift. A sparse budget grants deep, narrow access -- a few coordinates rewritten to any legal value, the rest untouched. A total-energy bound (L-2) allows either shape. Those are three different attackers with different real-world counterparts, so a robustness number quoted without both halves is quoting nothing.
go deeper
Be ready to say why an attack must be bounded at all, and to name what the two halves of the bound are: how change is measured and how much is allowed.
An interviewer expects you to map each norm family onto the attacker it describes -- broad and shallow, narrow and deep, or bounded total energy -- and to say what units the radius is in.
Show that you check whether the stated ball corresponds to a capability anyone actually has in the deployment, rather than accepting a number inherited from another input domain.
Own the position that a robustness claim is a claim about one attacker description, and that adopting somebody else's ball imports their assumption about who your attacker is.
## Why a budget exists at all An adversarial example is an input that somebody who does not own the model changed on purpose so the model reads it wrong. Without a constraint on that change the exercise is empty: an attacker permitted to rewrite the input arbitrarily can simply submit a genuinely different input, and the model's new answer is then *correct*, not fooled. Every evasion claim is therefore stated relative to a set of changes the attacker may make around the original input, and that set is almost always written as a ball -- a **norm** saying how the size of a change is measured, and a **radius** saying how much of it is allowed. The misconception this leaf exists to correct is treating that pair as a hyperparameter -- "we picked a small perturbation radius" -- as if it were a learning rate you could have set slightly differently. It is not a knob on your experiment. It is your written description of the attacker, and if you change it you have measured a different attacker. ## Three families, three attackers **Per-coordinate cap (an L-infinity ball).** Every coordinate of the input may move, but none by more than the radius. This describes broad, shallow write access: the adversary touches everything a little. Real counterparts are a calibration drift across a sensor array, a small uniform bias, or a market participant who nudges every one of their own postings slightly. **Sparse budget (a count of coordinates, L-0 style).** A few coordinates may be rewritten, possibly to any legal value, and the rest are untouched. This is deep, narrow access: one field of a record, one time step, one packet. The count is the budget and the magnitude is essentially unbounded. **Total-energy bound (an L-2 ball).** The total squared change is bounded. The allowance may be concentrated in one coordinate or spread thinly across all of them, so this family commits to neither shape and sits between the other two. These are not three parameterisations of one attacker. They are three attackers with different capabilities. A model evaluated under one of them has been evaluated against one of them. ## The radius is a quantity in somebody's units The radius is expressed in the units of the input *as the model sees it*. If the model consumes standardized values, a radius of a few hundredths is a few hundredths of a standard deviation -- and what that buys in the real world depends entirely on how wide that coordinate's distribution is. Two coordinates carrying different physical units under a single radius receive different real-world allowances, and nobody in the chain necessarily converted either of them back into a quantity a person can reason about. This is where budgets get borrowed rather than derived. A radius that became conventional in one input domain is a number calibrated to that domain's units and, often, to a *perceptual* proxy: the idea that a change under this size is one a human would not notice. Carried into a domain where nobody inspects the input at all -- a window of numbers feeding a forecaster, a flow record, a tabular row -- the perceptual justification has no validator, and the number names a size that has not been connected to any real capability. ## What the pair silently permits A budget forbids some things and permits everything else inside it, including shapes nobody pictured when the number was chosen. A per-coordinate cap across a long input window forbids any spike but permits *every* coordinate to move by the cap in the same direction -- a coherent, sustained shift that no single-point outlier check will see. A total-energy bound over the same window permits the opposite: the whole allowance concentrated on one coordinate. "Small" in one shape is not small in the other. The ball also fails to line up with what the attacker can actually do. Real inputs have sign constraints, granularity, legal ranges and ownership -- an attacker usually writes only a *sub-block* of the coordinates. So the ball and the attacker's reachable set overlap rather than nest: the ball is too generous where it hands the attacker coordinates they cannot write, and too tight where a perfectly legal value they *can* post sits outside the radius. ## How to say this under questioning State the pair before the number, every time: which norm, what radius, over which coordinates, in which units. Then say what attacker that describes. If you cannot name a real capability that the ball corresponds to, you have not stated a threat model -- you have stated an arithmetic convenience, and any robustness claim resting on it inherits that.
- Does a smaller radius always describe a weaker adversary?No. A sparse budget that lets two coordinates be rewritten to any legal value can be far stronger than a tight per-coordinate cap applied to all of them, even though the second sounds larger. Strength comes from the norm, the radius and which coordinates the attacker can actually write, taken together -- never from the number alone.
- If the attacker can only write some of the input coordinates, is a ball still the right description?Only if you restrict it to the coordinates they can write. A ball over the whole input hands them values other parties supply, which overstates their reach; at the same time a small radius can understate them, because one legal value they are entitled to post may sit far outside it. Say the coordinate scope alongside the norm and radius.
- What goes wrong when a radius convention is carried from one input domain into another?The number was calibrated to the first domain's units and often to a perceptual argument -- a change too small for a person to notice. In a domain where nobody looks at the input, that justification has no validator, and the radius ends up naming a size that has never been mapped to anything an attacker can actually do.
A burglary threat model is not "a small burglar". It is which doors they can reach and how much force they can apply -- and someone who can lightly jiggle every window is a different problem from someone who can take a crowbar to one.
saying these in an interview costs you the question
- Calls the perturbation radius a hyperparameter you tune down
- Quotes a radius without saying which norm
- Assumes a smaller radius always means a weaker attacker
- Treats every norm as the same attacker at different strengths
- Justifies the radius as 'imperceptible' with nothing checking that