skip to content

Why is the sum of two independent normal random variables exactly normal?

level: middleimportance: must knowfreq 55%

answer

  1. a transform turns addition into multiplication
  2. expectation of exp(t*X)
  3. independence factors the expectation
  4. normal MGF exponent: t and t^2
  5. collect exponents: means add, variances add

basics

~20 s

Moment generating functions multiply for independent variables, and the product of two normal moment generating functions is again a normal one. The sum is normal with the two means added and the two variances added.

solid answer

~50 s

The clean proof uses moment generating functions. The MGF is `M_X(t) = E[exp(t*X)]`, and for independent `X` and `Y` it factors: `M_{X+Y}(t) = M_X(t) * M_Y(t)`. A normal with mean `m` and variance `v` has `M(t) = exp(m*t + v*t^2/2)`, so multiplying two of them gives `exp((m1 + m2)*t + (v1 + v2)*t^2/2)` — again exactly the MGF of a normal, this time with mean `m1 + m2` and variance `v1 + v2`. Since an MGF that exists in an interval around zero determines the distribution uniquely, the sum *is* normal, not merely approximately so. The same argument works for Poissons, whose MGF `exp(rate*(exp(t) - 1))` multiplies to a Poisson with the rates added. Two cautions: the difference `X - Y` is normal with variance `v1 + v2`, not `v1 - v2`; and the argument needs independence, since two marginally normal but dependent variables can sum to something non-normal.

go deeper

for a junior

Recall the result and its parameters: adding two independent normals gives a normal, with the means added and the variances added. Remember that standard deviations do not add.

for a middle

Be ready to run the proof: define E[exp(tX)], state that it multiplies under independence, substitute exp(mt + v*t^2/2), and collect the exponent to read off the new mean and variance.

for a senior

Expect a probe on assumptions. Show you can produce a dependent counterexample where both marginals are normal but the sum is not, and explain that joint normality, not marginal normality, is what the result requires.

for a principal

Own the modelling implication. When component errors are correlated, the neat additive-variance arithmetic understates the spread of a total, so be able to argue how you would check that assumption before it underpins a risk or error budget.

## What the moment generating function is The moment generating function of a random variable `X` is `M_X(t) = E[exp(t*X)]`, a function of a real parameter `t`. It is useful for two reasons. First, it converts addition into multiplication: if `X` and `Y` are independent, then `exp(t*(X+Y)) = exp(t*X) * exp(t*Y)` and independence lets the expectation factor, giving `M_{X+Y}(t) = M_X(t) * M_Y(t)`. Second, it is a fingerprint. The uniqueness theorem says that if two variables have MGFs that agree on an open interval containing `t = 0`, they have the same distribution. Together, those two facts turn a hard convolution integral into an algebra exercise. ## The normal case A normal variable with mean `m` and variance `v` has `M(t) = exp(m*t + v*t^2/2)`. Take independent `X` with mean `m1` and variance `v1`, and `Y` with mean `m2` and variance `v2`. Then `M_{X+Y}(t) = exp(m1*t + v1*t^2/2) * exp(m2*t + v2*t^2/2) = exp((m1 + m2)*t + (v1 + v2)*t^2/2)`. Read the result: it has precisely the shape `exp(m*t + v*t^2/2)` with `m = m1 + m2` and `v = v1 + v2`. By uniqueness, the sum is normal with mean `m1 + m2` and variance `v1 + v2`. Note the word *exactly*: this is an algebraic identity that holds for two summands or two hundred, with no appeal to any large-sample argument. Two normals added give a normal, full stop. ## Variances add, standard deviations do not If `X` has standard deviation 3 and `Y` has standard deviation 4, the sum has variance `9 + 16 = 25` and therefore standard deviation 5 — not 7. The same holds for the difference. Writing `X - Y` as `X + (-Y)` and noting that `-Y` is normal with mean `-m2` and the same variance `v2`, we get `X - Y` normal with mean `m1 - m2` and variance `v1 + v2`. Subtracting means but adding variances is the single most common slip on this material: subtraction removes signal, but it accumulates noise from both sources. ## The same trick for Poissons A Poisson variable with rate `r` has `M(t) = exp(r*(exp(t) - 1))`. Multiply two independent ones: `exp(r1*(exp(t) - 1)) * exp(r2*(exp(t) - 1)) = exp((r1 + r2)*(exp(t) - 1))`, which is the MGF of a Poisson with rate `r1 + r2`. So counts from two independent streams pool into a single Poisson stream with the rates added — the reason you can aggregate independent event counts across shards or hours without leaving the family. The same machinery shows binomials with a common success probability add their trial counts, and that gammas with a common scale add their shape parameters. ## Where the argument needs care **Independence is essential.** The factorisation step is the only place independence enters, and it is load-bearing. Here is a clean counterexample. Let `X` be standard normal and let `S` be an independent random sign, `+1` or `-1` with probability one half each, and set `Y = S * X`. By symmetry `Y` is standard normal too. But `X + Y` equals `2X` when `S = +1` and equals `0` when `S = -1`, so the sum puts probability one half on the single value zero. A distribution with an atom is certainly not normal. Two marginally normal variables therefore need not sum to a normal; what suffices is that they are *jointly* normal, of which independence is the simplest special case. **Existence is not automatic.** An MGF may fail to exist for any nonzero `t` when the tails are heavy — the Cauchy distribution is the standard example. The remedy is the characteristic function `E[exp(i*t*X)]`, which always exists, multiplies the same way under independence, and carries the same uniqueness theorem. Every MGF argument in this area has a characteristic-function version that is technically safer. **Uniqueness is doing real work.** Showing that two functions agree is only useful because an MGF determines a distribution. Without the uniqueness theorem, matching MGFs would prove nothing about the shape of the sum. ## How to present it in an interview A strong answer is four steps: define the MGF; state that it multiplies for independent summands; substitute the normal MGF and collect the exponent; invoke uniqueness. If the interviewer pushes for an alternative, mention that the direct route is a convolution integral of two normal densities, which works but requires completing the square in the exponent — the MGF argument is the same computation in disguise, with the algebra made easy.

  • Why does this argument need the uniqueness theorem for moment generating functions?
    Because matching a function is only evidence if the function is a fingerprint. The uniqueness theorem says two variables whose MGFs agree on an open interval containing zero have the same distribution, which is what licenses the jump from 'the product looks like a normal MGF' to 'the sum is normal'. Without it the algebra would prove nothing about the shape.
  • Show the same argument for two independent Poisson variables.
    A Poisson with rate r has MGF exp(r*(exp(t) - 1)). Multiplying two independent ones gives exp((r1 + r2)*(exp(t) - 1)), which is the MGF of a Poisson with rate r1 + r2. So independent count streams pool into one Poisson stream with the rates added, and uniqueness again makes the conclusion exact rather than approximate.
  • Can two normal variables sum to something that is not normal?
    Yes, if they are dependent. Let X be standard normal and Y = S*X with S an independent random sign; Y is standard normal too, but X + Y is 2X half the time and exactly 0 the other half, so it has an atom at zero and cannot be normal. What guarantees a normal sum is joint normality, of which independence is the easy special case.
  • What do you do when the moment generating function does not exist?
    Switch to the characteristic function E[exp(i*t*X)], which exists for every distribution, still factors into a product for independent summands, and still determines the distribution uniquely. Heavy-tailed variables such as the Cauchy have no usable MGF, so the characteristic function is the technically safe version of the same argument.

saying these in an interview costs you the question

  • Subtracts variances for the difference of two normals
  • Adds standard deviations rather than variances
  • Claims any two marginally normal variables sum to a normal
  • Says moment generating functions add for independent variables
  • Treats the moment generating function as a density

context