Your ML lead wants every production model adversarially trained next year: what do you tell them?

## The mandate is the problem, not the method Adversarial training is a legitimate technique and there are deployments where declining to use it would be negligent. The defect in "every production model" is that it commits a fixed budget to one purchase across a portfolio whose models have completely different adversary pictures, and it does so without stating what the purchase costs or what it covers. A principal-level answer prices the purchase, states its coverage, and replaces the mandate with a test. ## What the purchase costs - **Clean accuracy, on all traffic.** Training the model to hold its output steady across a neighbourhood of each input rather than to fit the point costs accuracy on ordinary inputs. Every user pays that, including in deployments with no adversary. The cost is not evenly spread either: it lands hardest on rare classes and small subgroups, which is precisely the tail a business often cares about most. - **Training compute, as a multiple.** Meaningful adversarial training regenerates the worst case against the *current* weights at every step. The cheap alternative, augmenting with a fixed set of examples generated once before training, buys much less, because a model becomes robust to a fixed set quickly while remaining vulnerable to freshly generated ones. Any budget line that quietly substitutes the cheap version is buying the name, not the property. - **A permanent evaluation obligation.** Robustness is not a property you install and forget. It has to be measured against an adaptive attacker, and the number is meaningless without its columns: which attack, how many steps and restarts, which norm, which radius, and what access was granted. A high figure proves the attack that was run failed, not that the model is robust. Somebody has to own that measurement every release, and that is a standing cost. ## What the purchase covers Narrowly. Robustness bought against one change family transfers poorly to another and essentially not at all to a physical artefact, which respects no such budget. And on discrete inputs there is no small budget to grant in the first place: a whole number of days, a contractual pack size, a token in a document, a byte in a binary. Results obtained under a continuous perturbation budget do not carry over to those input spaces, so for a large share of a typical model portfolio, tabular and time-series and document-field models, the mandate would fund a technique whose threat model does not describe the deployment. ## The test that replaces the mandate Three facts, per model, each answerable in an afternoon: | Question | Fund robustness when | | --- | --- | | Who writes the inputs? | The writer is anonymous, free to create, or otherwise unattributable | | What does the output do? | It acts with no human on the costly path, and the action is not reversible | | What does a wrong output pay the writer? | Directly and immediately, in money or access | A model that answers yes three times is one where an adversary is motivated, cheap to be, and rewarded fast. An abuse, fraud or content classifier is the archetype: adversaries are free to create, the decision gates something they want, and the feedback is instant. Fund it there, evaluate it adaptively, and accept the accuracy bill knowingly. A model that answers no three times, such as an internal forecaster fed by contracted counterparties with a human approval threshold and a reversal window, should not be on the list at all. The finding that it faces no motivated adaptive adversary is a result of the work, not a failure to do it. ## What the same budget buys instead For the models that fail the test, the honest alternatives are cheaper and broader: - **Input provenance.** Knowing who wrote each value, being able to attribute it to an accountable party, and rejecting values that cannot be attributed. This raises the cost of being an adversary rather than the cost of a successful perturbation. - **Bounds and validation at ingestion.** Rejecting a field outside its contracted or historically observed range costs almost nothing and catches the crude version of both the attack and the accident. - **A reversible decision with a human on the costly path.** A value threshold and a reversal window cap the loss per wrong output regardless of what caused it. - **Watching the shape of what is submitted.** Sustained one-directional movement in one writer's declared fields, or output variance with no matching demand signal, catches the slow attack that stays under a per-order ceiling. Every one of those also covers non-adversarial failures: a broken feed, a units error, a genuine shock. Robustness training covers none of them. Per hour of engineering, the coverage is simply wider. ## How to say it to a lead Do not argue the technique. Offer the triage: run the three questions across the portfolio in a week, come back with the short list that passes, and fund those properly, adaptive evaluation included, rather than funding everything shallowly. Name what would change your mind per model, so the decision is re-openable: evidence of probing, a red-team result, or a product change that removes the human or the reversal. That converts a slogan into a defensible allocation with a written rationale, which is what survives the next budget review.