Why are regression residuals plotted against fitted values rather than against the observed outcome?

## The algebra behind the choice Fit a linear model by least squares with an intercept. Write each observation as `y_i = yhat_i + e_i`, where `yhat_i` is the fitted value and `e_i` the residual. Two facts follow from the normal equations, independently of whether the model is any good: - The residuals sum to zero. - The residuals are orthogonal to the fitted values and to every included predictor, so `Cov(yhat, e) = 0`. The second fact is why the residual-vs-fitted plot is the diagnostic of choice. It has a **known null appearance**: if the model's assumptions hold, the plot is a flat, even band, and it is flat by construction rather than by luck. Any systematic departure is therefore informative. ## What happens if you use the observed outcome instead Use the decomposition. Because `yhat` and `e` are uncorrelated, `Var(y) = Var(yhat) + Var(e)` and `Cov(y, e) = Cov(yhat + e, e) = Var(e)`. So if you regressed the residuals on the observed outcome, the slope would be `Cov(y, e) / Var(y) = Var(e) / Var(y) = 1 - R-squared`, and the correlation would be `Cov(y, e) / (sd(y) * sd(e)) = sd(e) / sd(y) = sqrt(1 - R-squared)`. Both are positive for any model that does not explain everything. A model with `R-squared = 0.25` gives a fake slope of 0.75 and a correlation of 0.87. The plot will look like a strong, tidy upward trend — and it means nothing. It is present for a perfectly specified model with normal, constant-variance, independent errors. The misreading it invites is specific and tempting: 'the model under-predicts high values and over-predicts low ones'. Someone then adds terms, transforms the outcome, or splits the data by outcome level, chasing a pattern that the arithmetic guarantees. ## The same reason makes residual-versus-predictor plots valid Orthogonality holds for every predictor included in the model, not only for the fitted values. That is why plotting residuals against each predictor in turn is the standard way to localise curvature: those plots also have a flat null, so a visible arc is real evidence about that variable. It is also why plotting residuals against a variable you did **not** include is informative in a different way. There is no orthogonality to protect you, so a clear relationship is evidence that the variable belongs in the model. ## Is a plot of observed against predicted useless, then? No — it is just a different tool. Observed against predicted is a **calibration** view: it shows how closely predictions track reality and over what range, and it is easy to explain to non-specialists. Two notes on reading it. First, in-sample, the least-squares line of `y` on `yhat` has slope exactly 1 and passes through the mean, again by construction, so a slope near 1 there is not evidence of anything. Second, the vertical scatter around the diagonal is the residual, which is exactly what the residual-vs-fitted plot displays with the diagonal flattened out — and flattening it is what makes changes in level and spread visible instead of being hidden along a steep line. ## Held-out data The orthogonality above is in-sample algebra: it comes from the fitting procedure. On new data the coefficients are fixed in advance, and there is no algebraic guarantee that the held-out residuals are uncorrelated with the predictions. If the model is genuinely right, they will be approximately uncorrelated because the expected error is zero everywhere. If the model is wrong — drifted, mis-specified, fitted on a different population — a trend appears. That makes a residual-versus-prediction plot on held-out data a genuinely sharper test than the same plot in-sample: in-sample, some structure was removed by the fitting itself, whereas out of sample nothing has been removed and the plot has to earn its flatness. ## How to say it in an interview Give the reason and the number: residuals are orthogonal to the fitted values by construction so the null picture is flat, while their correlation with the observed outcome is `sqrt(1 - R-squared)`, which is large for any ordinary model. Then add that the same orthogonality is why residual-versus-predictor plots are valid diagnostics, and that on held-out data the flatness is no longer automatic — which is precisely what makes it worth checking there.