You retrained a classifier on data produced by a backdoor-poisoning routine in an adversarial-robustness library. Which two numbers do you report, and why is the trigger success rate on its own a misleading result?

The instrument gives you poisoned rows. The **measurement design is entirely yours**, and it is where these assessments most often go wrong — usually in a way that flatters the attack. ### The two numbers, and what each answers **Clean accuracy** answers *would anyone have noticed?* Evaluate the retrained model on an untriggered held-out set and compare it against the same recipe trained on clean data. A backdoor that costs several points of top-line accuracy would be caught by ordinary model validation before release — so a headline success rate attached to a visibly degraded model describes an attack that never reaches production. Stealth is not a nice-to-have half of the claim; it is what makes the other half matter. **Trigger success rate** answers *does it work?* — the fraction of trigger-stamped inputs the model routes to the attacker's chosen target class. ### The denominator trap The success rate's numerator is easy; the denominator is where the number lies. Samples whose *true* class is already the attacker's target land in the target class on a clean model too. Leave them in the triggered test set and they count as successes with or without the trigger, inflating the reported rate by roughly that class's share of the set — about ten points on a balanced ten-class problem, and far more on an imbalanced one where the target is the majority class. **Exclude every sample whose ground-truth label is the target class** from the triggered evaluation, and state in the report which convention you used, because an unqualified success rate is not comparable to anybody else's. ### Run four evaluations, not two The minimal honest design is a two-by-two of {clean baseline model, poisoned model} against {clean test set, triggered test set}: | | clean test set | triggered test set | |---|---|---| | **clean baseline model** | the accuracy reference | control: does the trigger already misroute? | | **poisoned model** | the stealth number | the raw success number | The top-right cell is the one people skip and the one that saves you. If the clean baseline model *already* sends triggered inputs to the target class at some rate, part of your success number is the trigger pattern being an out-of-distribution artefact that any model handles oddly — not the backdoor. The effect attributable to the poison is the bottom-right minus the top-right, not the bottom-right alone. ### State the poison fraction, always A success rate is meaningless without the share of the training corpus you had to control to buy it. "Ninety-plus percent success at one percent poisoning" and the same rate at thirty percent are different findings, and only one of them describes an attacker anybody should plan against. The fraction is the bridge between a lab number and a threat model, and it must be a share of the **real** corpus, not of the subsample you experimented on. ### What it costs Every cell of that two-by-two above the diagonal is cheap — evaluation is minutes — but the *models* in it are not. You need at minimum two trained models (baseline and poisoned) under an identical recipe, and if you intend to claim a stealth cost you need several seeds of **both**, because a two-point accuracy gap is routinely within seed-to-seed variance for the same recipe on the same data. Four seeds of each is eight production trainings for one quotable configuration; at six GPU-hours per training that is roughly two GPU-days for a single row of the results table. Budget it before you promise the row. ### Where the number misleads Three readings, in descending order of how often they appear. First, a lone success rate with no clean-accuracy comparison — an attack that works and is obvious is not the finding it looks like. Second, the inflated denominator described above, which silently adds the target class's prior to every rate you quote. Third, a small clean-accuracy gap presented as *the stealth cost of the backdoor* when it is single-seed noise; the honest phrasing is "no accuracy cost was detected at the seeds we ran", which is a bounded, defensible statement. ### What I would check That the trigger is applied at evaluation exactly as the poisoning routine applied it during generation — same placement, same scale, same position relative to normalisation and augmentation — because a mismatch anywhere in that chain drives success to near zero and looks like robustness. That the clean and poisoned runs differ in nothing but injected data. That the target-class exclusion was actually applied to the triggered set, by checking the row count dropped by about the class share. And that the poison fraction quoted is a share of the production corpus.