What does it mean for a sample statistic to be an unbiased estimator of a population parameter?
answer
- a property of the recipe, not one number
- think across repeated samples
- look at the centre of the sampling distribution
- expected value versus the true parameter
- zero on average, for every theta
basics
~20 sAn estimator is unbiased when its expected value across all possible samples equals the true parameter. Its estimates are centred on the target: too high as often, and by as much, as they are too low.
solid answer
~50 sAn estimator is a rule you apply to a sample, so it is itself a random quantity with a distribution over repeated samples. It is unbiased if the mean of that distribution equals the parameter it targets: `E[theta_hat] = theta`, for every possible value of theta. The sample mean is unbiased for the population mean, because `E[xbar] = (1/n) * sum E[X_i] = mu` for any sample size and any distribution with a finite mean. Two clarifications matter in an interview. First, unbiasedness is a property of the *procedure*, not of the one number you computed — a single estimate that misses by a mile is not evidence of bias. Second, unbiased does not mean accurate: an estimator can be centred on the truth and still swing wildly from sample to sample, so variance has to be judged separately.
go deeper
Be ready to state the definition in one line and prove it for the sample mean using linearity of expectation. Say clearly that bias is about the average over repeated samples, not about any single estimate.
Explain why unbiasedness survives shifting and rescaling but breaks under square roots and other non-linear transforms, and give a concrete unbiased-but-useless estimator to show the property is weaker than it sounds.
Show that you judge an estimator on where it is centred and how much it moves, and that in real reporting you often accept a little bias for a large drop in variance rather than defending unbiasedness for its own sake.
Own the framing question: what loss are we actually minimising across all the numbers the organisation publishes? Decide when a centred-but-noisy metric is worse for decisions than a slightly shrunken, stable one.
## The setup A **parameter** is a fixed but unknown number describing a whole population — the mean height of adults in a country, the true click-through rate of a page. An **estimator** is a recipe that turns a sample into a guess at that parameter: take the sample mean, take the sample median, take the largest value observed. The number that recipe produces on one particular sample is an **estimate**. Keeping the two words apart is half the battle; interviewers listen for it. Because the sample is drawn at random, the estimator is a random variable. Draw a different sample and you get a different number. The distribution of those numbers over all the samples you might have drawn is the estimator's **sampling distribution**, and every property discussed here is a property of that distribution. ## The definition Write `theta` for the parameter and `theta_hat` for the estimator. The **bias** is ``` Bias(theta_hat) = E[theta_hat] - theta ``` and the estimator is **unbiased** when this is zero — not just for one convenient value of theta, but for every value the parameter could take. In words: if you could repeat the whole study endlessly and average all the estimates, that long-run average would land exactly on the truth. ## The worked case: the sample mean Let `X_1, ..., X_n` be a random sample from a population with mean `mu`. Each observation has `E[X_i] = mu`, and expectation is linear, so ``` E[xbar] = E[(X_1 + ... + X_n)/n] = (1/n) * (mu + ... + mu) = mu ``` This holds for **any** sample size — n = 3 is as unbiased as n = 3000 — and for **any** population shape with a finite mean, skewed or not. Unbiasedness of the sample mean needs no normality assumption at all, which surprises many candidates. Similarly, the sample proportion is unbiased for the population proportion, since a proportion is just the mean of zeros and ones. ## What unbiasedness is not **It is not a statement about your one sample.** You cannot look at a single estimate and declare the estimator biased. If a sample of 30 heights averages 171 cm when the population mean is 168 cm, that is ordinary sampling variation. Bias lives in the average over the infinity of samples you did not draw. **It is not accuracy.** Consider estimating the population mean by simply reporting the first observation you happen to see. Its expected value is `mu`, so it is perfectly unbiased — and it is a terrible estimator, because it is as noisy as a single data point no matter how much data you collected. Being centred on the target says nothing about how tightly the estimates cluster around it. Spread is a separate property and is judged separately. **It is not preserved by non-linear transformation.** This trips up strong candidates. The sample variance with the n-1 divisor is unbiased for the population variance, but its square root — the sample standard deviation — is **biased low** for the population standard deviation. Taking a square root is a concave operation, and averaging then transforming is not the same as transforming then averaging. Unbiasedness survives linear rescaling and shifting; it generally does not survive squaring, taking roots, inverting, or exponentiating. **It is not the same as sampling bias.** Statistical bias is a mathematical property of an estimator given that the sample was drawn as assumed. Sampling bias is a data-collection failure — surveying only people who answer the phone at 2 p.m. The sample mean is a perfectly unbiased estimator of the mean of the population you actually sampled from; if that population is not the one you care about, no amount of estimator theory rescues you. ## Why interviewers care Unbiasedness is the first vocabulary check in any estimation discussion, and the honest follow-up is always "so is unbiased always what you want?" The mature answer is no: unbiasedness is one desirable property among several, and it is routinely traded away for a large reduction in variance. What an interviewer wants to hear is that you know the definition precisely, can prove it for the sample mean in one line, and do not confuse being centred on the truth with being close to it.
- If the sample variance with the n-1 divisor is unbiased for the population variance, is the sample standard deviation unbiased for the population standard deviation?No. The square root is a concave function, so the average of the square roots is smaller than the square root of the average: the sample standard deviation is biased low. Unbiasedness is preserved by linear transformations such as shifting and rescaling, but not by non-linear ones like square roots, reciprocals or exponentials. In practice the downward bias is tiny except at very small sample sizes.
- Can you name an unbiased estimator of the population mean that no one would ever use?Report the first observation and ignore the rest. Its expected value is exactly the population mean, so it is unbiased at every sample size, yet its spread never shrinks — collecting a thousand points buys nothing. It is the cleanest demonstration that unbiasedness alone is a weak requirement: it constrains where the estimates are centred, not how tightly they cluster.
- Your survey oversamples one region and the sample mean misses the national average badly. Is the sample mean a biased estimator here?Not in the statistical sense. The sample mean is unbiased for the mean of the population the sample was actually drawn from, which is your oversampled one. What failed is the sampling frame, not the estimator. Fixing it means reweighting to the target population or repairing the design; swapping in a different estimator of the same wrong quantity changes nothing.
A bathroom scale that reads a random amount high or low but averages out to your true weight is unbiased. It is still useless for tracking a two-pound change, because unbiased says nothing about how much it wobbles.
saying these in an interview costs you the question
- Says unbiased means each estimate is close to the true value
- Claims one sample's error proves the estimator is biased
- Thinks unbiased implies best or minimum variance
- Assumes the sample standard deviation is unbiased for sigma
- Confuses statistical bias with biased data collection
- Believes the sample mean is only unbiased for normal populations