You are assessing backdoor risk with a library that emits poisoned training rows, and every configuration you test costs a full retrain of a production-sized model. How do you design the sweep so the finding holds up, without running hundreds of trainings?

Design the sweep **backwards from the sentence you want in the report**, because retrains are the scarce resource and every one you spend on a question the report will not answer is gone. ### Pick the one axis that changes the answer **Poison fraction** is almost always that axis, because it is the only knob that maps directly to attacker capability. "Controlling one row in a thousand of the ingest pipeline" and "controlling one row in three" are wholly different claims about who could do this and whether anyone should care. Trigger design, target class, architecture and optimiser are held fixed, named in the scope section, and left to a follow-up. A sweep across four axes at once produces a grid nobody can afford and a result nobody can interpret, because no two cells differ in one thing. ### Tier the compute Explore cheaply, confirm expensively. A reduced-scale proxy — a smaller model, a subsampled corpus, fewer epochs — is fine for finding roughly *where* the success rate turns on, which is a shape question. Then spend full-scale retrains on a handful of points bracketing that knee. **Say explicitly in the report which points were confirmed at production scale**, because a reader who cannot tell will assume all of them were, and a threshold that only ever existed on a one-tenth-size model is the kind of number that gets quoted back at you in a board deck. ### Spend seeds where the claim is Training variance is the quiet killer. Retrain the identical recipe on identical clean data twice and the accuracies differ; retrain a borderline poisoned configuration twice and the backdoor may take once and not the other time. So repeat **only the configuration you intend to quote**, together with its clean baseline, across several seeds, and report a spread rather than a single pair. Points far from the threshold — clear successes and clear failures — do not need repeats, and paying for them is how the budget evaporates. ### Keep the pipeline honest Inject **upstream** of the project's cleaning, dedup, label-validation and augmentation stages, and log how many poisoned rows survive to the optimiser. Rows appended straight into the tensor the training loop consumes bypass exactly the defences the organisation is already paying for, so the resulting number **overstates** risk: it describes an attacker who has been granted the one thing a real attacker has to earn. And if the stages do destroy the poison, that is a finding — a defensive one, often more valuable than the attack number, and considerably cheaper to deliver. ### What it costs, concretely Price the sweep before you start. A defensible minimal shape is roughly: six to eight reduced-scale explorations (cheap, hours), three or four full-scale bracketing points, and four seeds each of the quotable configuration and its clean baseline. That is on the order of a dozen production-sized trainings; at six GPU-hours apiece, about three GPU-days of compute and, more importantly, a week or more of an engineer's time — most of it spent not on the attack but on getting crafted rows into a pipeline that was never designed to accept them. Against a typical two-to-three week engagement, that is the majority of the technical budget, which is precisely why the axis choice has to be made deliberately rather than discovered halfway through. ### Where the numbers mislead The threshold from a single successful run at a borderline fraction is the headline error: with retrain variance in play, one success is an anecdote, and rerunning the *poisoning routine* with a new random seed changes nothing at all unless you also retrain — the weights are what carry the backdoor. A reduced-scale threshold quoted as a production threshold is the second. And the subtlest: a fraction expressed as a share of the exploration subset rather than of the real corpus, which can be off by an order of magnitude and always in the direction that makes the attack look easy. ### Cap the engagement, and pre-register the negative Fix the retrain budget and the stopping rule before the first run, and decide in advance what you will report if the sweep is inconclusive. "**No backdoor took at or below X percent poisoning, under this recipe, at these seeds**" is a legitimate and useful outcome with its assumptions attached. Quietly widening the sweep until some configuration finally succeeds is how a poisoning assessment becomes a month of GPU time and a claim nobody can reproduce. ### What I would check at the end That the clean baseline and every poisoned run differ in nothing but injected data; that surviving-poison counts were logged after filtering; that every quoted fraction is a share of the production corpus; and that each number in the table is labelled with the scale it was produced at.