What does the Cramer-Rao lower bound say about the variance of an unbiased estimator?

## The statement Let `T` be an estimator of a scalar parameter `theta` computed from `n` independent, identically distributed observations, and suppose `E[T] = theta` for all `theta` (unbiasedness). Under regularity conditions, the **Cramer-Rao lower bound** says `Var(T) >= 1 / I_n(theta) = 1 / (n * I_1(theta))` where `I_1(theta)` is the Fisher information in a single observation — the variance of the score, equivalently the expected negative curvature of the one-observation log-likelihood. The inequality generalises. If `T` is not unbiased for `theta` but estimates some smooth function, with `E[T] = psi(theta)`, the bound becomes `Var(T) >= psi'(theta)^2 / I_n(theta)`. And for a vector parameter, the covariance matrix of an unbiased estimator minus the inverse information matrix is positive semi-definite, which in particular bounds each variance on the diagonal from below. ## Where it comes from The proof is one application of the Cauchy-Schwarz inequality. The score `U(theta)` has mean zero, and differentiating the unbiasedness identity `E[T] = theta` under the integral sign yields `Cov(T, U) = 1`. Cauchy-Schwarz then gives `1 = Cov(T, U)^2 <= Var(T) * Var(U) = Var(T) * I_n(theta)`, and rearranging is the bound. That derivation also tells you exactly when equality holds: Cauchy-Schwarz is tight only when the two random variables are perfectly linearly related, so an estimator attains the bound precisely when `T - theta` is proportional to the score. This is the structural reason only a narrow family of models admits bound-attaining estimators. ## Attaining it, and not **Attained.** For Bernoulli data with success probability `p`, one observation carries information `I_1(p) = 1/(p(1-p))`, so the bound for any unbiased estimator of `p` is `p(1-p)/n`. The sample proportion has exactly that variance, so it is efficient in the strict finite-sample sense — no unbiased alternative can do better at any sample size. **Not attained.** For Normal data with unknown mean, the standard unbiased estimator of the variance `sigma^2` has variance `2*sigma^4/(n-1)`, while the Cramer-Rao bound for `sigma^2` is `2*sigma^4/n`. The estimator sits strictly above the floor. That is not a defect to fix — it reflects that no unbiased estimator here is a linear function of the score. **Bound does not apply.** The regularity conditions require the support of the distribution to be free of the parameter, so that you may differentiate under the integral sign. For observations uniform on `(0, theta)`, the support *is* the parameter, the derivation collapses, and estimators based on the sample maximum achieve precision that shrinks like `1/n` rather than the `1/sqrt(n)` the naive bound would suggest. Quoting a Cramer-Rao bound for such a model is a classic error. ## Two boundaries people trip over **Unbiasedness is a precondition, not a decoration.** The bound constrains variance only among unbiased estimators. A shrunken or penalised estimator accepts some bias and can have strictly smaller mean squared error than the Cramer-Rao floor. There is no contradiction: it is simply outside the class the theorem quantifies over. **The bound is about variance, not about achievability by any particular procedure.** It says what is impossible, not what a given method delivers. A high bound (little information) means no procedure will save you; a low bound does not promise your estimator reaches it. ## The asymptotic connection The bound is the exact-sample counterpart of the large-sample story: as `n` grows, the maximum-likelihood estimate is approximately Normal with variance `1/(n * I_1(theta))`, so it meets the bound in the limit even in models where no finite-sample estimator attains it. That is what people mean when they say maximum likelihood is asymptotically efficient. Notice the direction of the logic: the bound is a fact about the model and its information, and asymptotic likelihood theory says the maximum-likelihood estimate eventually saturates it. ## Interview framing A good answer states the inequality with the correct direction (variance is at least the reciprocal of information), names the two conditions (unbiased, regular), gives one example that attains it and one that does not, and closes with the asymptotic link. A weak answer flips the inequality, forgets unbiasedness, or asserts that every maximum-likelihood estimate attains the bound at finite `n`.