Why does adding all four region dummies plus an intercept break an OLS regression?

## The setup A region variable takes four values — North, South, East, West — and each observation belongs to exactly one of them. Encode all four as 0/1 indicator columns and put them into a regression that also has an intercept. The model will not fit, or will fit only after something silently disappears. ## Why it breaks The intercept is not a magic constant; it is a column of ones in the design matrix `X`. Because every row belongs to exactly one region, the four dummy columns satisfy ``` North + South + East + West = 1 for every row ``` That sum *is* the intercept column. So one column of `X` is an exact linear combination of four others. `X` is **rank-deficient**: its columns are linearly dependent. Ordinary least squares solves the normal equations `X'X b = X'y`. When `X` is rank-deficient, `X'X` is singular — it has no inverse — and the system has infinitely many solutions rather than one. This is **perfect** (or exact) multicollinearity, and it is categorically different from the everyday case of two predictors being merely highly correlated. High correlation inflates standard errors; a perfect linear dependency destroys identification altogether. ## What 'infinitely many solutions' looks like Suppose one solution has intercept 100 and region coefficients (10, 4, -2, 0). Add any constant `c` to the intercept and subtract the same `c` from all four region coefficients: intercept 105 with (5, -1, -7, -5). Every fitted value is unchanged, because every row picks up `+c` from the intercept and `-c` from its own region. The residuals, the residual sum of squares and R-squared are identical too. So the *fit* is perfectly well defined — the column space of `X` is unchanged, and the projection of `y` onto it is unique. What is not defined is the **decomposition** of that fit into an intercept plus region effects. That is what 'the coefficients are not identified' means: the data contain no information that would let you prefer one of those infinitely many splits over another. A candidate who says 'the model overfits' or 'the standard errors get large' has missed the point; nothing was estimated at all. ## The two clean fixes **Fix 1 — intercept plus k-1 dummies.** Drop one region, say North. The remaining columns are linearly independent, the model is identified, and the reading changes accordingly: the intercept is the expected outcome for North (with other predictors at zero), and each of the three surviving coefficients is that region minus North. The dropped level is the reference level. This is the default and the one that generalises, because as soon as the model contains a second categorical variable you cannot drop the intercept twice. **Fix 2 — no intercept, all k dummies.** Keep all four columns and remove the intercept. The model is again identified, and each coefficient is now that region's own expected outcome rather than a contrast. This reads nicely for a single factor, but it has costs: contrasts between regions now require differencing coefficients, the reported R-squared from a no-intercept fit is computed differently and is not comparable with an intercept model's, and the trick cannot be repeated for a second categorical predictor. What you must not do is both — dropping the intercept *and* a level leaves the model unable to represent the dropped region at all. ## Detecting it The symptom is unmistakable once you know it: a coefficient reported as missing or blank, an error about a singular or non-invertible matrix, or a fit that quietly returns one fewer coefficient than you supplied columns. Many fitting routines guard against the trap by dropping a redundant column automatically, which is convenient and dangerous — the model fits, and unless you read the output carefully you may not notice that the level you thought you were estimating became the baseline. The same trap appears in less obvious dress. Two categorical variables whose levels nest — a country dummy set and a currency dummy set where each country has exactly one currency — produce a dependency of the same kind. So does a set of dummies for month plus a dummy for 'quarter one' that is exactly the sum of three of them. Any time indicator columns are constructed so that some subset adds up to another column, identification is gone. ## The one-line answer to give The dummies for an exhaustive, mutually exclusive categorisation sum to the intercept, so including all of them alongside an intercept makes the design matrix singular and the coefficients non-identified; use `k-1` dummies with an intercept, or `k` without one.