In differentially private training, why must the added noise be scaled to the per-example clip bound to hide a record from an adversary?

## The quantity the noise has to cover Start from what the adversary is trying to do. They hold the released weights and want to distinguish the training run that included one particular record from the otherwise identical run that did not. The whole construction is designed so that those two runs produce outputs an adversary cannot reliably tell apart. At a single step, the difference between the two summed gradients is the contribution of that one record. Left alone, that difference is unbounded: a record can be an extreme outlier, or deliberately constructed to produce a huge gradient. Clipping each example's own gradient to a fixed bound before summing forces that difference to be at most the bound, for every possible record. That worst-case difference is the **sensitivity** of the summed update to one record, and clipping is how it is made finite and known. Noise is then added to the summed clipped gradient at a scale expressed **relative to that bound** — the standard deviation is a multiplier times the clip bound. This is the only sensible reference point, because the multiplier is what says how large the random perturbation is compared with the largest signal any one record could possibly have injected. ## Why the multiplier is the number that matters A useful consequence, and a common interview probe: **halving the clip bound while holding the multiplier fixed leaves the per-step privacy unchanged.** The absolute noise standard deviation halves, but so does the maximum influence it is hiding, and the ratio is unchanged. Conversely, a report quoting only an absolute noise standard deviation tells you nothing until you also know the bound it is relative to — the same number can be generous or negligible depending on the clip. What *does* change when you rescale the bound is utility, and it moves in two directions at once: - **A smaller bound** means more examples are rescaled down, so the update becomes a biased estimate of the true gradient — dominated by directions on which examples agree, with individually influential examples flattened. - **A larger bound** means less clipping bias, but the same multiplier now injects proportionally more absolute noise into each step. So the bound is not a free knob to tune for accuracy; it sets the noise scale at the same time. Practitioners typically look at the distribution of per-example gradient norms and choose a bound around the bulk of it, accepting bias on the tail, then choose the multiplier for the privacy target. ## Why neither operation works without the other **Noise without clipping.** With unbounded sensitivity, no finite noise level suffices. There is always some record influential enough that its contribution stands clear of the perturbation, and the guarantee has to hold in the worst case over records, not on average. **Clipping without noise.** The record's influence is now small and bounded, and it is also completely deterministic. Two runs differing by one record produce two different weight vectors, and small is not the same as indistinguishable. Randomness is what converts a bounded difference into one an adversary cannot resolve. This is the step people skip when they describe the mechanism as "we clip, so one row barely matters" — bounded influence is a precondition for the argument, not the argument. ## Where the noise goes, and where it does not The noise is added inside the training step, to the aggregate of clipped per-example gradients, at every step. It is not added to the data, not to the weights once at the end, and not to the model's answers at inference. Each of those alternatives breaks the accounting: the guarantee tracks the randomness in the sequence of updates the adversary can reason about, and a single perturbation somewhere else does not stand in for it. One more property worth naming because it is what makes the whole thing worth paying for: because the bound and the noise are defined against a worst-case record rather than against any particular attack, the protection is not attack-specific. It holds against membership tests, reconstruction attempts and extraction strategies that had not been invented when the model was trained, which is the reason a formal bound is preferred to an empirical robustness number in the first place. ## What a good answer sounds like "The clip bound is the sensitivity — the most one record can change the summed update — and the noise is calibrated to it, so the privacy-relevant quantity is the ratio, not the absolute standard deviation. Halve the bound at a fixed multiplier and privacy is unchanged, but you have added clipping bias. Without the clip there is no finite worst case to calibrate against; without the noise the bounded influence is still deterministic and still readable off the weights."