Clipping and Noise

## The claim under examination Somebody says: *"we clip gradients and add noise, so the model is differentially private."* Both operations named are real operations that appear in real training code. Neither of them, as usually implemented, is the mechanism that produces a per-record guarantee. Being able to say why, precisely, is the whole of this question. Fix the adversary first, because a privacy claim with no adversary in it means nothing. The adversary here **holds the released weights** — a downloaded checkpoint, or a model shared with the members of a consortium that pooled data to train it — and wants to decide whether one particular record was among the training rows. They do not need to break anything; they only need the trained parameters to depend detectably on the presence of that row. ## What clipping a batch's global norm bounds Ordinary training computes a loss over a mini-batch, backpropagates, and gets **one summed (or averaged) gradient vector for the whole batch**. Global-norm clipping rescales that single vector if its length exceeds a threshold. The quantity it caps is *the size of this step*. It is a stability control: it stops one pathological batch from throwing the run. What it does not do is limit the share of that step contributed by any one example. If a batch of five hundred rows contains one record whose gradient is enormous and four hundred and ninety-nine whose gradients roughly cancel, the clipped batch vector still points essentially where that one record pushed it. Remove the record and the direction changes visibly. The cap was on the sum, and a bound on a sum is not a bound on a term. ## What per-example clipping bounds The private construction changes the unit. Each training example's **own** gradient is computed, its norm is measured, and it is rescaled down to a fixed bound if it exceeds it. Only then are the clipped per-example gradients summed. Now the difference between the summed vector with the record and the summed vector without it is at most that bound — *by construction, for every possible record, including one crafted to be as influential as possible*. That worst-case, per-record ceiling is exactly the quantity the rest of the guarantee is built on, and it is why the guarantee holds against attacks nobody has published: it is a statement about how much the released weights can depend on one row, not a statement about any particular attack. It is also why this is expensive. An ordinary training step never materialises a per-example gradient; it gets the batch gradient directly and more cheaply. Buying the per-record bound means paying for the per-example quantity the fast path was designed to skip. ## Why the clip alone is not enough, and where the noise goes A capped contribution is still a contribution. If one record can move the update by up to some bound and nothing is random, an adversary who can reason about both possibilities can still see which way it moved. So noise is drawn at a scale tied to that bound and added to the **summed clipped gradient, during training, at every step**. The bound makes the record's maximum influence small; the noise makes the direction of whatever influence remains indistinguishable. Both halves are needed and neither works alone: unbounded sensitivity means no finite noise suffices, and bounded sensitivity with no noise leaves a deterministic signal. ## Why output noise is a different thing entirely Perturbing predictions happens after training, to a model whose parameters were already fitted on the record in question. It changes no weight. Three consequences follow: - An adversary holding the weights bypasses the output path completely, so the noise is not even in their way. - Even against a query-only adversary, independent noise on repeated answers can be averaged away unless the number of answered queries is itself bounded and accounted. - Nothing about it constrains how much the record influenced the parameters, which is the quantity a training-set guarantee is about. Output perturbation is a legitimate technique for releasing a *statistic*; it is not a way to make a trained model private after the fact. ## How to say it in an interview Name the unit. "Global-norm clipping bounds a batch; the guarantee needs a bound on one example, so the clip is applied per example before the sum. And the noise has to be added to that clipped training gradient, not to the model's answers, because the weights are what the adversary is reading." If you are then asked what the stability clip in the loop is doing, say it is a different operation with a different purpose and it neither helps nor harms the privacy argument.