In federated training, does keeping raw data on the device make the system safer?
answer
- Two properties, moving opposite directions
- Nothing arrives except a vector
- The corpus you could review is gone
- Enrolling is not reading
basics
~20 sNo. Keeping raw data on the device improves data privacy but makes integrity worse. The server now aggregates model updates it cannot inspect, so anyone who can enrol as a participant holds unauditable write access to the model.
solid answer
~50 sFederated training moves the trust problem rather than removing it. In a cross-device setup — say next-word prediction across millions of handsets — each client trains locally on its own text and returns only a model update, so the operator never sees the text. That is a genuine privacy gain. But it inverts the integrity story. In centralized training the operator holds the corpus and can sample it, re-label it, deduplicate it or hold parts out; in federated training the only thing that arrives is a vector of parameter changes, with nothing behind it to inspect. So enrolling as a client is not read access with a little write attached — participation *is* the write. What one participant is worth per round is the size of contribution the server will accept, and that multiplies by the identities they can enrol and the rounds before the next model release.
go deeper
Be ready to state the trade in one sentence: raw data stays local, and in exchange the server accepts contributions it cannot inspect. Know that enrolling as a participant is itself write access to the model.
Explain the mechanics that cause it — clients return parameter deltas, the examples behind them are gone by design, so sampling and re-labelling have nothing to operate on.
Show you can name what a participant is worth per round and what multiplies it, and that you would not read flat aggregate accuracy as evidence that nothing happened.
Own the framing that this is a deliberate trade of one property for another, and be able to say which team inherits the cost — privacy exposure falls, assurance about model contents becomes unavailable.
## What federated training actually is Instead of collecting examples into one corpus, the operator sends the current **global model** out to many clients. Each client trains a few steps locally on data that never leaves it, and returns an **update** — the difference between the parameters it received and the parameters it ended with. A server combines the updates from the clients sampled that round into a new global model and the cycle repeats. In the cross-device form the population is enormous — millions of consumer handsets running something like next-word prediction — and only a small cohort is sampled per round. The pitch is easy to state: raw text never leaves the phone. ## What genuinely improved Data minimization, and it is real. No central corpus is assembled, so there is no single store to breach, browse, subpoena or mis-retain. If the question is *who can read a user's sentences*, federated training answers it well. ## What got worse, and why it is structural rather than a gap Ask a different question — *what got written into the model, and by whom* — and the design has no answer available. In centralized training an adversary who wants to influence a model has to land rows in a corpus, a crawl, or a labelling queue. That is hard in a specific, useful way: an artefact exists. Somebody can sample it, deduplicate it, re-label a slice, hold a slice out, or compare a new batch against an old one. A degradation attack is diluted by corpus size, and a labeller looking at a flipped label may simply fix it. In federated training none of that machinery has anything to bite on. The only artefact is a vector of parameter deltas. The examples that produced it are gone by construction — that absence *is* the feature being sold. The operator cannot ask *what data produced this update?* because the honest answer is that the system was built so nobody could know. And within their own device, a participant is under no obligation to have trained on anything at all: they can submit whatever vector they like, and no reviewer exists to notice. So the correct summary is a trade, not an upgrade: **data privacy improved; contribution integrity got strictly worse**, because the server now accepts writes it cannot audit. ## Participation as write access, and what it is worth A useful way to hold it: the reach of an enrolled adversary is a product of three factors. - **The size of contribution the server accepts.** Averaging is a vote in which magnitude is the weight, so where the server bounds the norm of an accepted update this factor is capped at a known value; where it does not, the factor is unbounded. - **The number of identities enrolled.** Nothing about enrolment is a compromise — the adversary registers clients, they do not break into anything. - **The number of rounds** available before the next evaluation gate or model release. That product is the whole reach. Notice what it is *not*: it is not a detection capability, and none of the three factors gives the operator visibility into what was submitted. They price the write; they never review it. ## Claims that point the wrong way Three of these come up constantly, and each is backwards: - *Held-out accuracy stayed flat, so nothing happened.* Flat aggregate accuracy is equally consistent with an adversary who preserved it on purpose — a conditional behaviour attached to inputs they choose costs the average almost nothing. - *Clients attest their device, so contributions are trustworthy.* Attestation and signing establish **who** submitted and that the bytes were not altered in transit. They establish nothing about what the update encodes. - *No raw data is exposed, so the system is secure.* Exposure and integrity are different properties. The first improved; the second is the one this design spends. ## What to say in an interview Federated learning is a sound design for a real problem, and the answer that scores is not that it is dangerous. It is that it **relocates trust**: from an operator who can inspect a corpus to a population of strangers whose contributions arrive already computed. Once you say that, the follow-up questions — how much is one participant worth, how many identities does an adversary need, what evidence exists after the fact — all have somewhere to go.
- Does federated training remove the poisoning problem or relocate it?Relocate it. Write access moves from the corpus to the update channel. Centrally, an attacker must get rows past sampling, deduplication and re-labelling; in a federation they submit a vector nobody can open. The compensating control is therefore not review but bounding what one contribution is worth.
- The operator says an attacker cannot reach the training data. Is that true, and does it matter here?It is true that no central corpus exists to reach, and that is a real privacy gain. It does not help with integrity: an adversary controls their own device completely, so their own training data — or their decision not to train at all — is unreviewed by construction. The model still learns from what they returned.
- Which stakeholder is worse off after moving from centralized to federated training?Anyone who must answer for what the model contains. Privacy and legal exposure improve; the person triaging a suspicious round, or asked to certify that a released model is clean, loses the artefact their whole method depended on.
Accepting prefabricated wall sections instead of raw materials: you never expose your stockroom, and you also never see what is inside the walls.
saying these in an interview costs you the question
- Says federated learning is strictly safer than centralized
- Treats data privacy and model integrity as the same property
- Thinks client attestation proves what an update encodes
- Assumes the server can inspect a submitted update
- Reads flat aggregate accuracy as proof nothing was written