In adversarial training, how does the strength of the inner attack cap the robustness the model ends up with?

## Two searches, one radius Adversarial training is a nested problem. The outer loop fits weights; the inner loop, run for every batch, searches inside the stated perturbation region for the input the current model handles worst. The outer loop can only ever learn from **the points the inner loop actually returns**. If the inner search is bad at finding hard points, the model is being trained on easy ones and told they are the worst case. The two ends of that spectrum are worth naming precisely: - A **single-step** search moves once, as far as the radius allows, in the direction read off the loss with respect to the input. It is the cheapest possible inner search. - An **iterative** search takes many small moves and re-projects back inside the allowed region after each one, so it can follow a curved loss surface instead of committing to one straight line. At the same radius it finds substantially harder points. Restarts from different starting points inside the region raise the odds further. At an identical radius, these two return different points, and the second is much closer to the region's true worst case. ## Why the weak search caps the result If the loop only ever shows the model the point a single step reaches, the model learns to be right there. The fitted neighbourhood is not the ball; it is the thin set of points the weak search can reach. An evaluator running the same weak attack sees a high number. An evaluator running an iterative attack — at the same norm and the same radius — sees a much lower one. Nothing about the threat model changed between those two measurements: only the search did. This is why a robustness result is unreadable without **steps and restarts** alongside the norm and radius. A high figure proves that *the attack that was run* failed, not that the model is robust. Single-step training also has a specific documented failure: partway through training, robustness against iterative attacks can collapse abruptly while robustness against the single-step attack jumps — the model has learned to defeat that particular search rather than the perturbations it was standing in for. The literature calls that catastrophic overfitting. It is the sharpest illustration of the general principle: you become robust to the *procedure* you trained against, and only to the region insofar as the procedure explored it. ## The price of a stronger inner search Each inner step costs about as much work as a training step's own pass over the batch. A loop with k inner steps therefore runs at roughly **k+1 times** the cost of ordinary training, before restarts. Ten inner steps is an order-of-magnitude training bill; that multiple recurs on every retrain, every hyperparameter sweep and every model in the family. That cost is why inner steps are the first thing quietly reduced under a deadline, and why the first question to ask about a surprisingly cheap robustness number is how many inner steps and restarts produced it. The returns do flatten: past the point where the inner problem is nearly solved, extra steps move the measured robustness very little, so the defensible choice is the smallest step count at which the number stops improving — established by measuring, not assumed. ## Evaluation must be stronger than training A rule that follows directly: **evaluate with a strictly stronger search than you trained with**. Training with the iterative form and evaluating with the single-step form guarantees a flattering number. More steps, more restarts, and white-box access granted on purpose during evaluation, so the result does not silently measure your own obscurity rather than the model's fitted neighbourhood. If a stronger evaluation search collapses the number, that is information you needed before shipping — an attacker who holds the weights will run the stronger search, and they are attacking fixed final weights rather than the moving target the training loop chased, which is the easier problem. ## The compressed answer The outer model fits whatever the inner search finds, so the search quality is a ceiling on robustness at a fixed radius; a weak search produces a model robust to that search and not to the region; the cost is roughly linear in inner steps, which is why the number of steps and restarts belongs next to every robustness figure, on both the training and the evaluation side.