What is the difference between the weak and strong laws of large numbers?

## Two modes of convergence Both laws conclude that the sample mean `Xbar_n` of independent identically distributed draws approaches the population mean `mu`. They differ in what kind of approaching they assert. **Weak law — convergence in probability.** For every `eps > 0`: ``` P(|Xbar_n - mu| > eps) -> 0 as n -> infinity ``` Read this one `n` at a time. Fix a large sample size, run the experiment, and the chance of missing `mu` by more than `eps` is small. Nothing is asserted about how one particular infinite sequence of averages behaves over time. **Strong law — almost sure convergence.** ``` P( lim_{n -> infinity} Xbar_n = mu ) = 1 ``` Read this one *trajectory* at a time. Imagine writing down the whole infinite sequence `Xbar_1, Xbar_2, Xbar_3, ...` for a single run. With probability one, that sequence converges to `mu` in the ordinary calculus sense: for any band around `mu`, the sequence enters it at some point and never leaves again. ## Why the distinction is not pedantic Convergence in probability permits a sequence to keep failing, provided the failures become rare enough. It says the *chance* of a spike at time `n` shrinks; it does not say the spikes stop. Almost sure convergence forbids infinitely many spikes: on all but a probability-zero set of runs, only finitely many terms are far from `mu`. The canonical example of the gap is a sequence of indicator variables built from sweeping windows on the interval from 0 to 1. The first window is the whole interval; then two half-length windows sweep across it; then four quarter-length windows; then eight, and so on. Each variable equals 1 when a uniformly chosen point lands in the current window and 0 otherwise. The probability of a 1 is the window width, which goes to zero, so the sequence converges to 0 *in probability*. Yet the windows keep sweeping across the entire interval forever, so every point is inside a window infinitely often: no individual trajectory ever settles at 0. This sequence converges in probability but not almost surely — the exact behaviour the weak law tolerates and the strong law rules out. ## Which implies which Almost sure convergence implies convergence in probability. The strong law is therefore the stronger statement, and it implies the weak law. The reverse is false in general: convergence in probability does not imply almost sure convergence, as the sweeping-window sequence shows. ## Assumptions For independent, identically distributed draws: - The **strong law** holds whenever the expected value is finite. No variance assumption is needed. - The **weak law** holds under a finite expected value too, and follows immediately from the strong law. - The **elementary proof** of the weak law adds an assumption of finite variance, because it goes through Chebyshev's inequality: the sample mean has variance `sigma^2 / n`, so `P(|Xbar_n - mu| >= eps) <= sigma^2 / (n * eps^2)`, which vanishes as `n` grows. That extra assumption is an artefact of the proof technique, not a requirement of the result. A frequent interview stumble is to claim the weak law needs finite variance and the strong law needs more. It is the other way round in spirit: the strong law is the stronger conclusion, reached for independent identically distributed data under the same mild finite-mean condition, and the variance appears only to make an easy proof available. ## Does the distinction matter in practice? For most applied work, no — you have one finite dataset, not an infinite trajectory, and the weak law is the statement that actually describes your situation: with this much data, a large miss is unlikely. The distinction earns its keep in three places. 1. **Simulation and iterative estimation.** When you run one long simulation and watch a running estimate, you are looking at a single trajectory. The strong law is what justifies believing that *this* run settles, rather than merely that any given checkpoint is probably fine. 2. **Theory.** Proofs of consistency for estimators and algorithms often need trajectory-level convergence, and citing the wrong mode is a genuine error. 3. **Diagnosing conditions.** Asking which law applies forces you to ask whether the mean is finite and whether the draws are independent — the questions that actually determine whether anything converges. ## How to answer it State both modes precisely, note that almost sure implies in probability so the strong law is stronger, give the intuition (each fixed `n` is probably fine, versus each whole run eventually settles), and mention that for independent identically distributed draws a finite mean is enough for both. If you can describe a sequence that converges in probability but not almost surely, you have demonstrated you understand the difference rather than reciting labels.