Adversarial Robustness Tooling

A preprocessor defence transforms inputs before the model sees them. A postprocessor alters what comes back, usually the scores. A detector flags a sample as adversarial instead of classifying it. A trainer defence retrains the weights. A transformer hands you back a new, modified model object. Only trainer and transformer touch weights.

Adversarial accuracy is over every example you evaluated, flipped or not. The mean perturbation size is over only the examples the attack actually flipped; rows it failed on are dropped, not counted as huge perturbations. Two different denominators, so the two numbers are not about the same population.

The attack library implements the attacks and holds the parameters; the harness only drives it, doing target setup, batching, logging and output. So the library decides which attacks exist and how tunable they are. The harness decides how conveniently you run them, and may expose fewer attacks or fewer parameters.

The scan target wraps your endpoint as a callable: given a batch of samples it returns the model's per-class scores, or a label. You also declare the input shape and data type, the list of output classes, and a few seed samples the attack will perturb. It hands over queries only, never gradients or weights.

Multiply the number of evaluation examples by the attack's per-example query cap, times restarts, and add one clean pass to find which examples are classified correctly. Attacks spend the whole cap on examples they fail. Set the cap explicitly, pilot on ten examples, measure the real call count, then scale.

They come from the attack class's own constructor defaults, baked in so the library's documentation example runs fast on a laptop. They are demo settings, not assessment settings. A model that resists them has resisted only a short, weak search. Choose and record every strength argument yourself before reporting a robustness number.

The library only crafts tainted training data; it does not train anything. You have to mix those rows into the training set, retrain the model with your normal recipe, then evaluate it twice: normal accuracy on clean test data and the attack's success on the triggered inputs. The verdict costs a retrain, not one call.

When you wrap a model for an adversarial-example library you declare the legal minimum and maximum for input values. The attack clips every generated example back inside that box, so pixels stay in range. It is one global range over all features, and it knows nothing about what any individual feature means.

Three levels. White-box classes need loss gradients through the model. Score-based black-box classes need the full confidence vector per query. Decision-based classes need only the predicted label. You establish which from the class's documented requirement and the arguments its constructor insists on; these libraries generally reject a model that cannot supply what the class needs.

They form an ordered chain applied in the order you attach them, so the second sees the first one's output. Each declares whether it runs at training, at inference, or both. A defence that runs only at training is absent when the attack queries the model, so the run measures a pipeline nobody serves.

A predict-only wrapper maps a batch of inputs to model outputs. A gradient-exposing wrapper must additionally return the derivative of the loss with respect to the input, at the same input the model sees. Stub that and gradient attacks still run, but they follow a fake signal and report robustness that only measures your stub.

Because it is a mean over an empty set. No example was flipped, so nothing entered the average and the helper returns zero as a degenerate default. On this scale small means fragile, so the failure case prints at the alarming end. Check the attack success count before reading the number at all.

The wrapper gives you one uniform way to point attacks at a model plus a repeatable loop: consistent target definition, batch runs, logging and result output you did not write. You pay with its dependency pins, its supported platform list, and a parameter surface narrower than the library's own attack classes.

Count per attack, then add up. Queries are roughly seed samples times iterations times calls per iteration, and a black-box attack spends many calls per step estimating a direction or probing a boundary. Do not trust the arithmetic alone: put a counter in the target's callable, run one attack on one sample, and extrapolate.

Attacks call methods on your wrapper. Gradient attacks need a gradient method, which a forward-only HTTP wrapper cannot implement, so they error or are unavailable. You are left with query attacks: score-based ones that read the returned probabilities, decision-based ones needing only the top label, and transfer from a local surrogate.

More iterations let the search refine longer inside the same bound. A step size too large for the bound overshoots and one too small never reaches its edge. More restarts re-launch from fresh random starting points, so one unlucky start is not read as robustness. Each knob multiplies GPU time roughly linearly.

Report two: accuracy on a clean, untriggered test set against the unpoisoned baseline, and the trigger success rate measured only on samples not already in the target class. Success rate alone hides a poisoned model that lost obvious accuracy, and it is inflated by inputs the clean model already sent to the target.

The mask freezes coordinates: features you mark immovable keep their original values, so the search only touches the ones you allow. It says nothing about the movable features' own rules, such as integer counts, one-hot exclusivity, or a field derived from another. Those you check yourself after the library returns its examples.

Assume the attack broke rather than the model became robust. Re-tune the attack against the defended pipeline, raise iterations and restarts, and check that an unbounded perturbation still drives accuracy to zero. Compare against a query-based attack and a transfer attack. Confirm the defence really sits inside the wrapper you attacked.

The wrapper exposes the bare model, so the attack optimised a tensor the live service never receives. The resize and re-encode step sits outside the wrapper and destroys the perturbation. Fix it by moving the whole serving preprocessing chain inside the wrapper, so the attack's input is the uploaded file rather than the tensor.

Because the run only tested the attack at the settings the wrapper exposed. An attack with an untuned step size, iteration count or perturbation budget failing says nothing about robustness. Before publishing, check which parameters the layer passes through, sweep the ones that matter, and record the exact settings tested.

Assume the harness is broken first. Re-run with the bound relaxed until the attack must succeed; if it still fails, the wrapper or the loop is wrong. Then sweep iterations and restarts upward and watch the success rate. Also check a cheap non-gradient baseline: if it beats the gradient attack, the gradients are unreliable.

It tells the library the legal bounds of an input, so every perturbed candidate is clipped back into them. Declare 0-255 when your model is actually fed 0-1 and the attack searches a space your service never accepts, so the perturbation size you report means nothing.

It treats every column as a continuous pixel and nudges all of them. You get rows with fractional counts, several one-hot columns partly set so no single category is encoded, values outside legal ranges, and a changed account age. The label flips, but no attacker could submit that row, so the success rate measures nothing actionable.

The pipeline now has three outcomes, not two: correct, wrong, and rejected. Decide up front how a rejected sample counts, for adversarial and for clean inputs separately. Report the false-positive rate on clean data beside the detection rate, because the threshold is tunable and a detection rate alone hides what it costs benign traffic.

It drags a whole transitive stack into your environment: its pinned framework and attack-library versions can conflict with what your training or serving code needs. Contain it by giving the wrapper its own isolated environment or container and moving data across as files, rather than importing it into an existing project.

It fixes comparability: everyone runs the same fixed attack set at the same effort and threat model, so numbers can be ranked and nobody quietly under-configures. It does not fix relevance. It measures one threat model on one dataset through a required interface, not your inputs, your preprocessing pipeline or the attacks your product actually faces.

You supply two things: a wrapper around the victim that answers queries, and an untrained substitute model you chose and configured. The library queries the victim over inputs you provide, labels them, and fits your substitute. It returns your object, now trained. Architecture, query pool and the query bill are all yours.

Constraints are filters inside the loop: a transformation proposes candidate sentences, each constraint rejects the ones that violate it, and the search only ever sees the survivors. They are what keeps a rewrite readable and meaning-preserving. Loosen them and the success rate rises, because you are now counting rewrites that changed the sentence.