How does variance reduction cut Monte Carlo error without buying more draws?

## Why bother Monte Carlo error is roughly `sigma/sqrt(n)`, where `sigma` measures how much a single simulated run varies. Buying accuracy through `n` is brutal: four times the compute to halve the error. Variance reduction attacks `sigma` instead, and a technique that halves `sigma` is worth exactly as much as quadrupling the run count, usually for a fraction of the cost. ## Antithetic variates Simulations are driven by uniform random numbers. Run the simulation once on a stream of uniforms `U`, then run it again on the mirrored stream `1-U`, which is equally valid uniform randomness. Average the two outputs into one paired observation. The variance of an average of two variables is Var((X1 + X2)/2) = (Var(X1) + Var(X2) + 2*Cov(X1, X2)) / 4 With independent runs the covariance is zero and you get half the variance of a single run, exactly what two draws should buy. With antithetic pairs the covariance is negative whenever the output is a monotone function of the driving uniforms, so the paired variance is strictly less than that, and the same compute buys more precision. Monotone dependence is common: a longer service time makes a queue longer, a higher uniform makes an inverse-transform draw larger. The failure mode is precise and worth naming. If the output is a symmetric function of `U`, so that `f(U) = f(1-U)`, the pair is perfectly positively correlated and the two runs contribute exactly as much information as one. You have paid for two runs and received one. For strongly non-monotone outputs, antithetic pairing can therefore increase variance per unit of compute. ## Control variates Suppose alongside the quantity of interest `X` you can also compute, in the same simulated run, a related quantity `Y` whose true expectation `E[Y]` you know exactly from theory. Then Z = X - c*(Y - E[Y]) has the same expectation as `X` for any constant `c`, because the correction term has mean zero. Its variance is Var(Z) = Var(X) - 2*c*Cov(X, Y) + c^2 * Var(Y) Minimising over `c` gives the optimal `c* = Cov(X, Y)/Var(Y)`, and substituting it yields Var(Z) = Var(X) * (1 - rho^2) where `rho` is the correlation between `X` and `Y`. So the reduction depends entirely on correlation: 0.5 removes 25% of the variance, 0.9 removes 81%, 0.99 removes 98%. Weakly correlated controls are barely worth the code. Two practical points. `E[Y]` must be known exactly, not estimated from the same run, or the correction reintroduces the very noise it is meant to cancel. And `c*` is itself estimated from the simulated output, which adds a small bias at tiny sample sizes; with thousands of runs it is negligible, and a pilot run can supply `c` for the main run to avoid the issue entirely. ## Common random numbers When the question is not "what is the win rate" but "is design B better than design A", simulate both on the identical seeded stream of random numbers. Whatever luck the stream contains hits both designs, so it largely cancels in the difference `B - A`, and the comparison becomes far sharper than two independently seeded runs would allow. Formally, the variance of a difference is `Var(A) + Var(B) - 2*Cov(A, B)`, and positive induced correlation shrinks it. This is where seeding stops being merely a reproducibility habit and becomes a statistical tool. Note the distinction plainly: a fixed seed on a single run buys reproducibility and nothing else; the same seed deliberately shared across two compared configurations buys precision on their difference. The technique requires the two designs to consume the random stream in the same order, which in practice means giving each source of randomness its own generator so a change in one design does not shift every downstream draw. ## Importance sampling, briefly When the quantity of interest is dominated by rare outcomes, sampling from the natural distribution wastes almost every run on irrelevant cases. Importance sampling draws instead from a distribution that visits the interesting region often, then reweights each draw by the ratio of the true density to the sampling density to keep the estimate unbiased. It can cut the variance of a rare-event estimate by orders of magnitude, and it can also blow the variance up spectacularly if the sampling distribution has lighter tails than the target, which makes a few enormous weights dominate the average. ## Choosing among them Try the cheapest applicable technique first. Common random numbers costs almost nothing when comparing configurations and is nearly always right. Antithetic variates costs one line for a monotone simulation. Control variates need a known-mean companion quantity, which is a modelling insight rather than a code change. Importance sampling needs care and a good proposal, and belongs to rare-event work. Always verify the reduction empirically by comparing the spread of the reduced estimator against the plain one on a pilot run, since a technique applied to the wrong integrand can quietly make things worse. ## What to say out loud Explain that error is spread over `sqrt(n)` and that these techniques attack the spread, give antithetic pairing with its monotonicity condition, give control variates with the `1 - rho^2` formula, and distinguish reproducibility seeding from common random numbers.