For heteroscedastic data, when do you prefer weighted least squares over robust standard errors?

## Two different problems, two different fixes Heteroscedasticity causes two distinct injuries, and it is worth separating them because each correction addresses only one. 1. **Broken inference.** The conventional standard-error formula assumes one common error variance, so under unequal variances it is inconsistent and every t-statistic and interval built from it is wrong. 2. **Lost efficiency.** OLS gives every observation equal weight. Under unequal variances the low-noise observations carry more information about the line than the high-noise ones, so OLS is no longer the minimum-variance linear unbiased estimator - a better estimator exists. Robust standard errors fix (1) and ignore (2). Weighted least squares fixes (2) - and, if the variance model is right, fixes (1) as a side effect, because the weighted fit is homoscedastic in the transformed space. ## What robust standard errors buy The sandwich estimator plugs the observed squared residuals into the middle of the general variance expression instead of assuming a single variance times the identity. Its virtues are exactly its modesty: it commits to no shape for the variance, changes not one digit of the point estimates, and remains consistent under essentially any pattern. Variants differ only in small-sample correction - HC1 applies a simple degrees-of-freedom scaling, while HC2 and HC3 additionally adjust each squared residual for how strongly the fit was pulled toward that observation, with HC3 the most conservative and the usual recommendation for modest samples. Its limitations are also worth stating. It is an **asymptotic** argument: in genuinely small samples the sandwich estimator is itself noisy and can be anti-conservative, which is what HC3 partly compensates for. And it makes no attempt to recover efficiency - the intervals are correct but wider than they had to be. ## What weighted least squares buys If `Var(e_i) = sigma^2 / w_i`, minimising the weighted sum `sum w_i * (y_i - fitted_i)^2` produces the generalised-least-squares estimator, which under those weights is BLUE. The intuition is direct: an observation with a quarter of the variance carries four times the information about the line, so it deserves four times the weight. The gain is real and, when the variance ratio across rows is large, substantial - genuinely narrower confidence intervals rather than merely differently computed ones. Unlike robust standard errors, **WLS changes the point estimates**. Both estimators remain consistent for the same parameter when the model is correctly specified, so a large gap between the OLS and WLS coefficients is itself a diagnostic - it usually signals a mis-specified mean function or effects that genuinely differ across the population, since OLS and WLS then converge on different weighted averages of those varying effects. ## When are the weights actually known? This is the whole question, and the honest answer is: less often than textbooks imply. The defensible cases are: - **Aggregated rows.** Each row is a group mean built from a known number of underlying units. The variance of a mean over `n_i` units is `sigma^2 / n_i`, so `w_i = n_i` follows from arithmetic, not from a guess. City-level averages built from different city populations are the canonical case: weighting by population size is not a modelling choice, it is a correction for how the rows were manufactured. - **Known measurement precision.** Instruments or vendors that report a per-observation uncertainty give you the weights directly. - **Design weights.** A sampling design with known inclusion probabilities determines the weights up front. When the variance instead has to be estimated from the data - fit OLS, model the squared residuals, invert the fitted values into weights - you have moved into feasible generalised least squares. That can pay off when the variance relationship is strong and simple, but you have introduced estimation error into the weights themselves, and the theoretical efficiency guarantee is now conditional on a model you fitted from the same data. ## What goes wrong with wrong weights A useful reassurance and a useful warning. Weighting by something exogenous - unrelated to the error - keeps the estimator **consistent** even if the weights are badly chosen; you simply forfeit the efficiency gain and may end up worse off than plain OLS. What you do *not* keep is valid model-based inference: the weighted-model standard errors assume the weights are correct, and if they are not, those standard errors are wrong in the same way conventional OLS standard errors were wrong to begin with. The consequence is the practical protocol: **weight if you can justify the weights, and report heteroscedasticity-robust standard errors on the weighted fit anyway.** That combination gives you whatever efficiency the weights genuinely deliver and inference that survives being wrong about them. It is strictly better than choosing one and hoping. One genuine hazard: weights that depend on the outcome, or on anything the outcome influenced, break the exogeneity that keeps WLS consistent. Weights must come from the design or from predictors, never from the response. ## How to answer in the room Lead with the criterion rather than a preference: do I have a defensible variance model? Give the aggregated-data case as the example where the answer is yes, name efficiency as the thing WLS buys and agnosticism as the thing robust standard errors buy, and close with the belt-and-braces combination. Candidates who declare one universally superior are usually reciting; the interesting answer is the condition that decides.