Why prefer studentised deleted residuals over standardised residuals when flagging regression outliers?

## Three residuals, one row For observation `i` in an OLS fit with `n` rows and `p` estimated coefficients, there are three quantities people loosely call "the residual". **Raw residual.** `e_i = y_i - yhat_i`. Simple, in the units of the response, and treacherous as a comparison device, because its variance is not the same for every row: ``` Var(e_i) = sigma^2 (1 - h_ii) ``` A row with high leverage `h_ii` drags the fitted surface toward itself, so its raw residual is squeezed toward zero. Ranking rows by raw residual therefore systematically under-ranks exactly the rows most able to distort the fit. **Standardised (internally studentised) residual.** ``` r_i = e_i / (s * sqrt(1 - h_ii)) ``` where `s` is the residual standard error from the full fit. The `sqrt(1 - h_ii)` factor corrects the leverage-induced shrinkage, which is genuine progress: now all the `r_i` have roughly comparable spread. People commonly eyeball `|r_i| > 2` or `> 3` as worth a look. **Studentised deleted (externally studentised) residual.** ``` t_i = e_i / (s_(i) * sqrt(1 - h_ii)) ``` where `s_(i)` is the residual standard error from a model estimated *without* observation `i`. Everything is the same except the ruler. ## Why the ruler matters The defect in `r_i` is circular reasoning. `s` is computed from all the residuals including `e_i`. A badly discrepant observation therefore inflates `s`, and inflating `s` shrinks `r_i` — the point pushes down its own measured extremeness. With a single bad row in a small sample this can be enough to keep it under an informal cutoff. Using `s_(i)`, an estimate the suspect row never touched, removes the circularity: the observation is judged against how well the *rest* of the data are fitted. This is the same leave-one-out logic that underlies influence diagnostics generally. Ask what the model would look like if this row had never existed, then ask how surprising the row is under that model. ## The reference distribution The practical payoff is calibration. Under the standard assumption of independent normal errors, the studentised deleted residual follows a **t distribution with `n - p - 1` degrees of freedom**. The arithmetic is direct: the deleted fit uses `n - 1` observations and `p` parameters, leaving `n - 1 - p` degrees of freedom for its error estimate. The internally studentised `r_i` has no such clean distribution — it is a bounded quantity, not a t variate, which is why the `|r| > 2` habit is a rule of thumb rather than a test. One caution about using that distribution as a test. You are typically not asking about one pre-chosen row; you are scanning all `n` rows and reacting to the largest. That is a multiple-comparison problem, and the usual fix is a Bonferroni adjustment: compare the largest `|t_i|` against the `alpha/(2n)` tail of the t distribution rather than the `alpha/2` tail. Without that correction, a few rows past `|t| = 3` in a dataset of ten thousand is exactly what normal errors would produce anyway. ## They rank the same, so why bother? A fair objection: `t_i` is a monotone increasing function of `r_i`, ``` t_i = r_i * sqrt((n - p - 1) / (n - p - r_i^2)) ``` so the two orderings of the rows are identical. Nothing changes about *which* row is most extreme. What changes is the interpretation of the number. The internally studentised residual is bounded by construction and has no reference distribution, so a value of 3.4 means whatever your habit says it means. The deleted version can be compared to a t distribution and, with the multiplicity adjustment, converted into a defensible statement about how surprising the most extreme row is. When the transformation blows up — `r_i^2` approaching `n - p` — that itself is the signal of one dominating row. ## How it connects to influence Studentised residuals are the discrepancy half of influence, not influence itself. Combine one with the leverage and you get an influence measure: DFFITS, for instance, is `t_i * sqrt(h_ii / (1 - h_ii))`, the deleted residual multiplied by the lever-arm factor. So the correct reading is that a large `|t_i|` says "this response disagrees with what the rest of the data imply", and only pairing it with leverage tells you whether that disagreement moved the fit. ## The short interview version Raw residuals are unfairly small for high-leverage rows; standardised residuals fix that but are measured against a ruler the suspect row helped build; deleted residuals fix the ruler too and come with a `t_{n-p-1}` reference distribution, at the cost of needing the leave-one-out error estimate — which has a closed form, so it costs nothing in practice.