A federated round shifted the global model oddly, then four rounds looked ordinary — adversary or client population?

## The situation One round of a cross-device federation — next-word prediction over millions of handsets — produced a global model that behaved oddly on something you noticed. The next four rounds looked unremarkable. Somebody has to decide whether this was an ordinary property of a shifting client population or an enrolled adversary, and write a disposition. ## First, name what you do not have There is no stored batch behind any update, and there never was: the design's whole point is that the examples stayed on the devices. So the investigative move you would reach for centrally — pull the rows, look at them, re-label a sample — does not exist here. Saying this out loud early is not defeatism; it is what stops the triage from spending a week looking for an artefact that was never created. ## What you do have, and what each thing bounds **Update-size statistics.** The distribution of contribution norms in the round, and the fraction that hit whatever ceiling the server applies. A spike in that fraction says some submissions were far larger than typical. It does *not* say who or why: a client version that changed local step counts produces the same shape. **Cohort composition.** Which populations were sampled — locale, client version, device class, time of day. A shift that tracks a coherent population attribute is the honest explanation, and it is the one worth ruling in first because it is by far the more common cause. **Per-slice evaluation.** Aggregate held-out accuracy is nearly useless here. It stays flat under a targeted change by design, and it moves for a dozen boring reasons. Accuracy broken out by slice — locale, input length, vocabulary region — tells you *where* something landed, which is the difference between a population shift and a change concentrated somewhere nobody's data lives. **Probes against the released global model.** This is the strongest evidence you can generate, because the model is an artefact you control and can query as often as you like. If the odd behaviour is conditional on particular inputs, it reproduces on demand there, independent of rounds and sampling. **The reach arithmetic.** Ask whether the observed shift is even purchasable: given the norm the server accepts and the cohort size, how many identities would it take to produce it? If the answer is implausibly large, that is real evidence toward a population explanation. If it is one or two, the shift tells you nothing about intent. ## The intermittency trap "It happened once and not in the four rounds after" feels like a flake, and here it is not informative at all. Clients are **sampled** each round. An adversary holding a handful of identities appears in a cohort only occasionally, so intermittency is the expected signature of a small attacker and of an unusual honest sub-population alike. Frequency of appearance is not severity, and a finding that reproduces once in five attempts is not thereby a lower-severity finding — it is a finding you have to reproduce against the model rather than against the rounds. ## The disposition you can honestly write Be careful about the direction of every sentence in the ticket: - Four clean rounds bound **the checks you ran** in those rounds. They do not bound what the model contains. - Flat aggregate accuracy shows a broad degradation did not occur. It is equally consistent with an adversary who preserved it. - A probe that did not fire bounds **the probes you wrote** — one behaviour, on the inputs you thought to try. - No identity is ever established from an update alone. At most you can say a contribution was atypical in size. So the honest disposition is usually: population explanation ruled in or not, reach arithmetic stating how much an adversary would have needed, probe results with their coverage stated, and a decision about whether the model ships — not a verdict about who did it. If the evidence cannot separate the two causes, say that; the design does not owe you a separation. ## What changes the next investigation The useful output of a round like this is usually not attribution but instrumentation: retain the norm distribution and ceiling-hit fraction per round, retain cohort composition, evaluate per slice rather than in aggregate, and keep a probe suite that runs against every candidate release. None of those inspect a contribution — nothing can — but together they turn "something looked wrong once" into a claim with a shape.