Robustness & Defense Techniques

Because a model fits a fixed set of perturbed inputs within a few epochs and is then vulnerable to fresh ones. Adversarial training re-solves the attack against the weights as they are now; generating once up front is only augmentation.

It measured that one search stopped finding bad inputs, not that they stopped existing. A defence can make the signal an attacker steers by unusable while the same misclassified transactions remain reachable by a different search.

It stops an attacker whose perturbation was fitted against the bare classifier and never had to survive the cleaning step. An attacker who knows the step is there fits a perturbation through it and pays only extra optimisation steps.

The detector is another trained model with its own decision boundary and its own mistakes. An attacker aware of it crafts a single message that reads as ordinary mail to the detector and still fools the classifier behind it.

A standard suite runs attacks written before the defense existed, so a high score can mean the defense disturbed the attacker's search rather than stopped the attack. Evidence needs an attack designed by someone holding the defense's description.

Robust accuracy of 60% means one specific attack, run at one stated effort inside one stated edit budget, failed on 60% of the tested inputs. It bounds that attack, not the model's resistance to a better-resourced adversary.

Ask: robust to what? Adversarial training buys resistance only inside the perturbation family and radius it trained on. An attacker stays free to work in a different metric, or to change pose and occlude part of the face, which no per-pixel budget bounds.

All of it. Training a model to stay correct across a whole neighbourhood of each input costs clean accuracy on every request served, while the robustness pays off only on the rare manipulated request inside the trained-for budget.

Only a party who can write into the model's inputs. Here that is a supplier authoring lead-time, pack-size and price fields on documents the forecaster reads, with no query access and no scores returned.

The attacker searches one allowed set of edited messages for a point on the wrong side of the classifier's boundary and the ordinary side of the detector's. Two conditions in one budget means more search, not a wall.

An adaptive attack is designed after reading the defense. The adversary is assumed to know the mechanism and its parameters, and aims the attack at the quantity the defended pipeline actually decides on, rather than at the model underneath it.

Because adversarial training makes a model resist the worst case inside one fixed set of allowed changes. "Every coordinate moves a little" and "a few coordinates move a lot" are different sets, and points in the second sit far outside the first.

Of the smoothed classifier: the derived function that answers by majority vote over many noise-perturbed copies of the input. The network you trained is not the certified function, and serving it alone carries no guarantee at all.

The model only fits the worst case its inner search finds. A one-step search finds a poor one, so the model resists that search yet falls to a stronger search at the same radius. More steps buy a higher floor at proportional cost.

Exactly the attackers whose whole move is a per-pixel change no larger than the chosen radius. That robustness transfers poorly to sparse or energy-bounded perturbations, barely at all to a camera-captured artefact, and falls off steeply just outside the radius.

It means the white-box result is an artefact of a broken search, not a robustness property. An adversary holding weights can always imitate a weaker one, so a strictly weaker attacker outscoring them is logically impossible unless the stronger attack's gradient signal is unusable.

They replace the non-differentiable stage with a smooth stand-in when computing their search direction, and the natural stand-in is the identity because a content-preserving transform barely changes its input. The cost is extra optimisation steps, offline and cheap.

Not from those numbers. Robust accuracy ranks two models only when both faced the same edit budget, the same access class and at least as strong an attack. Optimisation steps alone can move such a figure by tens of points.

No. More data and capacity narrow the gap without closing it: staying correct across a neighbourhood of every input is a strictly harder objective than being correct at the point, and that capacity is not free.

It takes the payoff, not the input manipulation. The supplier can still move the forecast, but a wrong quantity becomes money only if it survives the window and the human-approval threshold, so they must also defeat whoever looks.

That the search is broken at every radius, not that the model is robust. With no limit on how far an input may move, an attacker can hand the model a transaction that genuinely belongs to the other class, so success should approach total; anything left standing is the optimiser failing.

A false-reject and re-image rate that lands on your hardest real parts, plus a cost increase for an adaptive attacker. The window between destroying a perturbation and destroying the defect signal is narrow, and the attacker works inside whatever survives.

Ask which attack produced it. A catch rate from attacks that did not know the detector existed shows those attacks were caught, not that the detector holds. Ask for the detector-aware number and the false-alarm rate.

Only that these designs, from this vantage, at this effort, did not break it. A failure to break is bounded by who tried and how hard, so the report states access, days spent and designs abandoned - never robust.

Ask for the figure as a curve over optimisation steps and restarts, run until it stops falling, plus the attack access class and the stated edit budget. Ten steps is a stopping point, not a result.

Almost nothing. The figure prices an attacker who edits pixels within a magnitude budget. Someone standing at the gate is bounded by viewing angle and how much of the face they may cover — units the defense never trained in and cannot report on.

Ask for what makes the two numbers comparable: the edit set and its size, the strength of the attack run and whether it was the training attack, per-slice clean scores, and the share of live impressions manipulated inside that edit set.

Almost nothing yet. Without the norm and radius, the confidence level, the abstention rate and which function was certified, the figure is unordered. Ask for those, then write a sentence bounded to exactly what was measured.

Robustness is an expensive, narrow purchase, so make the mandate a per-model test. Fund it where an unattributable writer profits from a wrong output; elsewhere the same hours buy more input provenance and a reversible, reviewed decision.