What bounds how hard an adversary poisons a retrained ranker whose alert thresholds they cannot see?

## The setting Take a ranking model retrained on a stream of logged user interactions. The team watches three things: a training loss curve, an offline replay score against the previous model, and a per-segment quality report that management reads. An adversary controls a pool of ordinary user accounts and can therefore write interaction events into the corpus that the next retrain consumes. They can read the product from outside. They cannot read the dashboard. ## What actually binds them The interesting fact about this leaf is that the adversary's limit is *someone else's instrument*. In most attack settings the constraint is theirs: a perturbation radius, a query bill, a share of clients. Here it is the defender's alert threshold and the granularity of the report — how far a number may move before somebody investigates, and which slices are broken out at all. Those two together set a ceiling on how strong a write may be per retraining cycle. And they bind exactly one thing. They bind **strength**. They do not bind **existence**: there is no level of alerting that makes an in-budget write impossible, because an in-budget write is by definition one the alert does not fire on. ## Uncertainty as the defender's real asset The attacker does not know where the line is. They can infer coarse things from outside — that a large, sudden shift in what the product recommends would be noticed by users, that a total collapse would obviously be caught — but they cannot resolve the threshold to a number. Two consequences follow: 1. **Rational play is conservative.** Overshooting once is the single event that ends the campaign and burns the account pool. Undershooting only costs time. An attacker who values persistence therefore stays well under whatever they guess the line to be, which is materially weaker than pushing to the line. 2. **The asymmetry is real and unusually cheap.** Nothing about the attack model changes if the threshold is tighter; what changes is the attacker's uncertainty and therefore their margin. This is one of the few places in adversarial ML where not publishing a parameter buys something durable rather than buying obscurity that evaporates on first contact. ## What the constraint converts into, rather than prevents With per-cycle strength capped, the attacker still has three dials, and this is the cost statement a red-teamer should be able to give: - **Share of the stream.** More controlled accounts contributing events means more in-budget influence per cycle. - **Dwell time.** A weak write repeated across many retraining cycles can accumulate, because each cycle's corpus still contains the earlier writes unless something removes them. - **Narrowness.** Aiming at a slice the report does not break out keeps the observable movement small by construction, rather than by restraint. All three trade time and scale for stealth. None of them is defeated by lowering the alert threshold; lowering it raises the bill and pushes the campaign slower and wider, which is a real result and should be claimed as exactly that. ## How to report this as a finding A red-team finding here has an awkward shape, because the honest conclusion is a cost curve rather than a yes/no. The useful form is: *the monitoring caps per-cycle effect at roughly the alert threshold; achieving the objective under that cap required this many accounts and this many retraining cycles; the campaign is detectable only if it overshoots or if the affected slice is reported separately.* That gives the reader the two things they can act on — the number of accounts, and which slices are reported — without pretending an in-budget campaign was blocked. ## The traps - **Claiming the threshold is a boundary.** It is a cost control. Boundaries stop attacks; cost controls decide which attackers bother. - **Assuming the attacker knows the threshold.** Assume white-box access to *weights* when you evaluate robustness, so you do not measure your own obscurity — but a monitoring threshold is an operational secret with real value, and modelling the attacker as omniscient about it throws away the analysis of the margin they must leave. - **Reading a long-quiet dashboard as evidence.** A campaign designed to stay under an alert produces a quiet dashboard as its normal operating condition. Duration of quiet is not evidence of absence; it is consistent with both hypotheses. - **Confusing this with drift.** A gradual slide in a monitored number has an enormous space of ordinary causes. The question here is what your instrument bounds, not how to attribute a movement to an adversary.