Why does adversarial training regenerate the worst case inside its perturbation ball against the current weights each step?

## The objective being solved Ordinary supervised training asks the model to be right on each training input. Adversarial training asks for something strictly stronger: be right *everywhere inside a small region around* each training input. The region is not vague — it is stated as a **norm and a radius**, for example "every pixel may move by at most r" (an L-infinity ball) or "the total energy of the change is at most r" (an L-2 ball). That pair is the whole promise the method is making, and it is chosen before training starts. Because you cannot enumerate a region, the training loop **approximates it by its worst point**. At every step it runs a search inside the ball for the perturbed version of each input that the model, *as it is at that moment*, scores most badly on. It then trains on that perturbed input **with the original example's label** — the label never changes, and that is the lesson being taught: everything within radius r of this photo is still the same category. ## Why the search must be repeated The worst case is not a property of the image. It is a property of **the image and the current weights together**. Change the weights and the point in the ball that hurts most moves. So the loop is a chase: the model improves on the perturbations found last step, the search finds different ones this step, and the two co-adapt. This is precisely what the common wrong answer misses. A team that generates a batch of adversarial examples once — before training, or from some other released model — and folds them into the training corpus has built **data augmentation**. The observable result is characteristic and easy to reproduce: after a few epochs, accuracy on *that saved set* is excellent, and accuracy under perturbations re-solved against the finished weights is close to an undefended model's. The model learned those images, not the neighbourhood around them. The gap between those two numbers is the cleanest evidence that the regeneration step was skipped. And the attacker never uses the saved set. Someone holding or approximating the shipped weights solves the search fresh, against the *final, fixed* parameters — an easier problem than the moving target the training loop faced. ## Why random perturbations are not a substitute A frequent follow-up is why you cannot simply add random noise of the same magnitude and skip the search. The reason is volumetric: the ball is enormous and the directions that flip a confident classifier are a vanishing fraction of it. Random noise of a magnitude that a targeted, structured change flips the label with essentially never does anything to a trained model. An adversarial perturbation is a **direction**, not noise, and it has to be found by searching rather than sampled. ## What it costs, and what a single-step search buys Each step of the inner search costs roughly the work of one more pass over the batch. A loop that runs k inner steps therefore costs on the order of k+1 times an ordinary training step, and that multiple — not the algorithm's difficulty — is the reason teams water the method down. A single-step inner search is the cheapest honest version and is genuinely much better than nothing, but the strength of that inner search caps the robustness that comes out; the model can only fit the worst case the search actually located. ## What the method does and does not deliver - It delivers **measured** resistance to perturbations of the norm and radius trained for, against a search of comparable strength. It is empirical, not a guarantee. - It delivers nothing outside that ball. Robust accuracy at the trained radius says nothing about a slightly larger radius, a different norm, or a change that is not a bounded perturbation at all. - It costs clean accuracy and it costs compute, and both bills land on all traffic while the benefit only applies against an adversary who stays inside the trained region. ## How to say it in an interview The compressed version: adversarial training fits a neighbourhood instead of a point; the neighbourhood is defined by a norm and a radius; the worst point in that neighbourhood depends on the current weights, so it is re-solved every step; a set generated once is augmentation and the model outgrows it. If you can also say what the regeneration costs — a multiple of training compute set by the number of inner steps — you have answered the question and the follow-up in one breath.