Running an Assessment

They come from the attack class's own constructor defaults, baked in so the library's documentation example runs fast on a laptop. They are demo settings, not assessment settings. A model that resists them has resisted only a short, weak search. Choose and record every strength argument yourself before reporting a robustness number.

The library only crafts tainted training data; it does not train anything. You have to mix those rows into the training set, retrain the model with your normal recipe, then evaluate it twice: normal accuracy on clean test data and the attack's success on the triggered inputs. The verdict costs a retrain, not one call.

When you wrap a model for an adversarial-example library you declare the legal minimum and maximum for input values. The attack clips every generated example back inside that box, so pixels stay in range. It is one global range over all features, and it knows nothing about what any individual feature means.

Attacks call methods on your wrapper. Gradient attacks need a gradient method, which a forward-only HTTP wrapper cannot implement, so they error or are unavailable. You are left with query attacks: score-based ones that read the returned probabilities, decision-based ones needing only the top label, and transfer from a local surrogate.

More iterations let the search refine longer inside the same bound. A step size too large for the bound overshoots and one too small never reaches its edge. More restarts re-launch from fresh random starting points, so one unlucky start is not read as robustness. Each knob multiplies GPU time roughly linearly.

Report two: accuracy on a clean, untriggered test set against the unpoisoned baseline, and the trigger success rate measured only on samples not already in the target class. Success rate alone hides a poisoned model that lost obvious accuracy, and it is inflated by inputs the clean model already sent to the target.

The mask freezes coordinates: features you mark immovable keep their original values, so the search only touches the ones you allow. It says nothing about the movable features' own rules, such as integer counts, one-hot exclusivity, or a field derived from another. Those you check yourself after the library returns its examples.

Assume the harness is broken first. Re-run with the bound relaxed until the attack must succeed; if it still fails, the wrapper or the loop is wrong. Then sweep iterations and restarts upward and watch the success rate. Also check a cheap non-gradient baseline: if it beats the gradient attack, the gradients are unreliable.

It fixes comparability: everyone runs the same fixed attack set at the same effort and threat model, so numbers can be ranked and nobody quietly under-configures. It does not fix relevance. It measures one threat model on one dataset through a required interface, not your inputs, your preprocessing pipeline or the attacks your product actually faces.

You supply two things: a wrapper around the victim that answers queries, and an untrained substitute model you chose and configured. The library queries the victim over inputs you provide, labels them, and fits your substitute. It returns your object, now trained. Architecture, query pool and the query bill are all yours.

Constraints are filters inside the loop: a transformation proposes candidate sentences, each constraint rejects the ones that violate it, and the search only ever sees the survivors. They are what keeps a rewrite readable and meaning-preserving. Loosen them and the success rate rises, because you are now counting rewrites that changed the sentence.

Treat it as censored data, not a model property. Check the queries-to-success distribution on the examples that worked, whether the perturbation was still shrinking when the cap hit, and whether errors, throttling or duplicate queries burned the allowance. Then re-run a stratified subsample at a much larger cap.

Retries and repeated probes spend billed calls. Throttling and latency spend wall-clock, which limits how many examples you finish per day. Variability is worst: a boundary answer that flips makes the search oscillate. Handle it by voting over repeated probes, which multiplies the budget, or requesting a deterministic setting if the endpoint offers one.

Sweep one axis that changes the answer, the poison fraction, on your real training recipe. Explore cheaply on a smaller model or data subset, then confirm only the interesting points at full scale. Repeat the borderline configuration across several seeds, because retrain variance can be bigger than the effect you are claiming.

Run the library's output through a validity predicate you write, before you score anything. Keep only rows a real applicant could actually submit, re-query the model on those, and report misclassified-and-valid over examples attempted, together with the share you discarded. The 91% was computed before that filter and overstates what is reachable.

There is no right split, only a stated one. Wide and shallow covers many examples at a low cap and mostly measures the cap. Narrow and deep gives credible per-example results with wide error bars. Usually: one cheap screening pass, then deep runs on a stratified subsample, holding budget back for re-runs.

Define two tiers. A screening tier may use cheap settings, and its results may only report hits, never robustness. A publishable tier requires a versioned configuration — attacks, effort, threat model — plus evidence the success rate plateaued and a forced-success sanity artefact. Same configuration for every model, so numbers stay comparable.

It shows the attack works against what you gave it: full probability outputs, no rate limit, no monitoring, and a query pool close to the training distribution. It does not show the deployed endpoint is exploitable. Restate it as production questions: what the endpoint returns, what the queries would cost, and whether anyone would notice.

Filter afterwards for a cheap first read: it is a few lines, but the search optimised in a space you then discard, so survivors are partly luck and the rate is loose. Project inside the loop when the number must mean something: it costs custom code, and you are no longer running a stock attack.