Why must the delta in an (epsilon, delta) privacy guarantee sit far below one over the dataset size?

## Where delta sits in the definition The pure form of the guarantee has one parameter. For neighbouring datasets — identical except for one record — the probability that the training procedure lands in any given set of outputs differs between the two worlds by at most a factor of `e^epsilon`. The approximate form adds a second term: at most `e^epsilon` times, **plus delta**. Epsilon is multiplicative and bounds how much the two worlds can differ; delta is **additive** and bounds the probability mass for which nothing is being promised at all. That asymmetry is the whole answer. A multiplicative slack degrades gracefully — a slightly larger epsilon means a slightly better-informed adversary. An additive slack does not degrade at all: it is a share of outcomes carved out of the guarantee, and the definition places no constraint on how bad those outcomes are. ## The construction that makes it concrete The standard illustration is deliberately absurd and it is what an interviewer wants to hear. Consider a procedure that, instead of training anything, selects a delta-sized share of the training records and publishes them exactly as they are, doing something harmless the rest of the time. For the records it does not publish, the two worlds are identical. For the ones it does, the guarantee has already spent its additive slack. The procedure satisfies the definition for **any** epsilon, including zero — and it is a catastrophic disclosure for the individuals it named. Nothing about real mechanisms works this way; the point is what the definition permits, and therefore what a number in a report fails to exclude. If delta is comparable to one over the number of records, the bound leaves room for a construction that exposes roughly one person outright. Multiply delta by n and you get the scale of what the slack could cover: on four million rows, a delta of one in a hundred thousand leaves room for about forty records' worth of unbounded failure. ## The convention, and how to check it Practitioners therefore require delta **far** below one over the dataset size — often stated as one over n raised to a power greater than one, or simply pinned at a cryptographically small value such as one in a billion for datasets in the millions. The reasoning is not aesthetic. Below one over n, the additive slack is smaller than the per-record scale the guarantee is about, so it cannot swallow a record whole. The check a reviewer performs on any privacy line is arithmetic: - Find n, the number of training records the number was computed over. - Compute one over n. - Compare delta to it. If delta is at or above that, the guarantee is close to vacuous however good epsilon looks, and the correct response is to say so rather than to read the epsilon. A report that gives epsilon and omits delta cannot be checked at all, and it also cannot be compared with any other report, because a mechanism can trade a smaller epsilon for a larger delta. ## Why the second parameter exists at all A fair follow-up is why anyone accepts the complication. Pure single-parameter guarantees are achievable, but the mechanisms that fit iterative training — additive Gaussian-type noise, combined with the accounting used when each step touches a random subsample — naturally produce a two-parameter statement, and they produce much tighter, more useful epsilons than pure mechanisms do at the same accuracy. Delta is the price: a genuinely tiny probability of the promise failing, bought in exchange for a model somebody can actually deploy. That is a reasonable trade at one in a billion. It is not a reasonable trade at one in a thousand. ## The direction of the claim The error to avoid is treating delta as a rounding term, a numerical tolerance, or a convergence threshold. It is none of those. It is the probability that the entire epsilon promise, against the adversary trying to tell whether a record was used, does not apply — and the definition is silent on what happens in that case. An epsilon reported without its delta is half a statement, and an epsilon reported beside a delta nobody checked against the dataset size is a statement nobody has verified.