What does constraining injected training records to look in-distribution cost the attacker who writes them?

## Reframing the filter as an exchange rate The useful mental model is not *does the screen catch the poison* but *what does the screen charge for*. A sanitization stage that drops records scoring high on unusualness imposes a constraint on any attacker who wants their records to reach training: whatever else those records do, they must look like plausible members of the corpus. That constraint has a measurable price, and the price is the answer to this question. ## Why looking ordinary reduces per-record leverage A fitted model is a compromise across the records it saw. How much any single record can pull that compromise depends on how far it sits from the rest and how the loss treats it. A record with extreme values, or a label that contradicts everything nearby, pulls hard - which is precisely why careless poison works with few records, and also why it is easy to spot. Force that record to sit inside the observed distribution and both properties change at once. It no longer stands out, so the screen keeps it; but it also no longer pulls hard, because it now resembles the records already pulling in the ordinary direction. The attacker has traded effect for survival. The shortfall is made up the only way it can be: more records. So the poison budget stops being *how extreme may my records be* and becomes *how many records can I get written*. On a network-intrusion model retrained on captures from the segment it monitors, that second budget is the interesting one, because the attacker's vantage is simply the ability to emit traffic that lands in the next capture. There is no query limit and no per-record cost beyond the traffic itself. Whether the raised bill is a real defence depends entirely on how many records that attacker can produce over how many retraining windows, and that is a number the defender is supposed to estimate rather than assume. ## The dilution asymmetry, which is where candidates go wrong A very common answer is *our corpus is enormous, so a handful of records cannot matter*. That is true for one kind of goal and false for the other. - **Degradation of general accuracy** works by shifting the overall fit, so it competes against every honest record in the corpus. Its required volume genuinely scales with corpus size, and a growing corpus dilutes it. - **A conditional behaviour keyed to inputs the attacker chooses** does not compete against the corpus. It asks the model to learn one additional, narrow association that nothing else in the data contradicts. The number of records that takes behaves as roughly an absolute count rather than a fraction, so a corpus ten times larger dilutes it very little. In-distribution constraint raises the price of both, but it raises them from different starting points, and a defender quoting corpus size as protection is implicitly assuming the first goal. ## What the filter does and does not bound The defensible statement after a sanitization stage is a conditional one: | The screen bounds | The screen does not bound | | --- | --- | | Records lying outside the observed distribution | Records constructed to lie inside it | | The cheapest form of the attack | The attack's existence at any budget | | How extreme a single record may be | How many ordinary-looking records may be written | Notice the shape: every row on the left is about the records, and every row on the right is about the attacker. That is the gap the question is aiming at. A screen is a property of your data pipeline; a budget is a property of whoever can write into it, and only the second determines whether the raised price binds. ## Where the raised price actually bites The price is a genuine defence in exactly one situation: when the write volume the attacker needs exceeds what they can produce without being noticed by some other mechanism. Two cases are worth separating. If write access is rationed - a small number of contributing sources, a per-source volume that is itself monitored, a corpus that only accepts records through a channel someone owns - then multiplying the required record count by a large factor can genuinely take the attack out of reach. If write access is effectively unmetered, as it is on a segment anyone can emit onto, multiplying the count by a large factor changes how long the attacker spends and nothing else. This is why the sanitization discussion so often ends up somewhere else: the filter's output is a price, and the interesting engineering question is who is able to pay it. That is a question about write access and volume, not about scoring rules. ## Answering it in an interview Say the cost plainly - per-record leverage - then say what it is exchanged for - volume - then name the two conditions that decide whether the exchange helps: how much the attacker can write, and which goal they hold. An answer that stops at *the poison gets weaker* has given half of it.