How much more data does it take to halve the width of a confidence interval for a mean?

## The square-root law Write the interval out: ``` xbar ± t(n-1) × s / sqrt(n) ``` The full width is twice the half-width, `2 × t × s / sqrt(n)`. Of the three factors, the critical value is fixed once you pick a confidence level, and `s` is a property of the population you are sampling — it does not shrink because you collected more data. Only `sqrt(n)` responds to effort, and it sits in the denominator. So: ``` width ∝ 1 / sqrt(n) ``` Solve for the sample size that gives a target half-width `h`: ``` n ≈ (critical value × s / h)^2 ``` The target half-width is *squared* in the denominator. That single square is the whole story: halving `h` multiplies n by 4, cutting it to a third multiplies n by 9, cutting it to a tenth multiplies n by 100. ## Working the numbers Suppose a sample of 100 gives a 95% interval of width 4. To reach width 2 you need about 400 observations; to reach width 1, about 1,600; to reach width 0.4, about 10,000. Going the other way explains why tiny pilot samples are so uninformative — dropping from 100 to 25 observations does not double the width by a little, it doubles it exactly. A useful way to describe this to non-specialists is in terms of marginal value. The move from 25 to 100 observations and the move from 400 to 1,600 observations both halve the width, but the second costs 1,200 extra observations and the first costs 75. Every doubling of precision is four times more expensive than the last one. ## The two caveats **The spread is estimated too.** You plan with a guess at `s` — from a pilot, from historical data, from a defensible upper bound. The realised interval uses whatever `s` the new sample produces, which is random. Plan with a conservative (larger) value for `s` if hitting the target width matters, because underestimating it leaves you short. **The critical value moves at small n.** If you are using t, quadrupling the sample also increases the degrees of freedom, and the t value falls toward the normal value. Going from n = 5 to n = 20, for example, the 95% t value drops from 2.776 to 2.093 — a 25% reduction in its own right — so the width shrinks by more than the square-root law alone predicts. Past a few dozen observations this second effect is negligible and the clean 4x rule holds. ## What more data will not fix This is where a strong answer separates itself. The square-root law describes *sampling noise only*. It says nothing about: - **Bias.** If the sampling frame excludes part of the population, a larger sample gives you a narrow interval tightly wrapped around the wrong value. The interval will look more authoritative and be no closer to the truth. - **Dependence.** The `sqrt(n)` term assumes n independent observations. If observations cluster — repeated readings on the same unit, users sampled from the same few groups — the effective sample size is smaller than the row count, and the interval is narrower than it deserves to be. - **Measurement error in the instrument itself.** Averaging removes random measurement noise but not a systematic offset. So the honest framing of the 4x rule is: *four times the data halves the noise-driven part of the uncertainty, and does nothing to the rest.* Once bias dominates, collecting more data is the wrong investment entirely, and the money is better spent fixing how the data are gathered. ## Does it apply to a proportion? The Wald form for a proportion has half-width `z × sqrt(p(1-p) / n)`, which carries the same `1 / sqrt(n)` behaviour, so the 4x rule transfers as a planning heuristic. Two differences are worth flagging: the spread term `p(1-p)` is largest at `p = 0.5` (where it equals 0.25) and shrinks toward the extremes, so the worst-case plan uses 0.5; and near 0 or 1 the binding problem is not the sample size but whether the normal approximation behind the formula holds at all.