What is a propensity score, and why match on it rather than on the raw covariates?

## The problem the score solves In an observational study nobody randomised anything. Units that received the treatment differ systematically from units that did not: they are older, larger, more engaged, further along, or simply more likely to have been offered it. A raw comparison of outcomes therefore mixes the effect of the treatment with the effect of those differences. The obvious repair is to compare like with like — pair each treated unit with an untreated unit that has the same covariate values. This is exact matching, and it works beautifully with two or three covariates. It collapses as soon as you have twenty. The number of distinct covariate cells grows multiplicatively with the number of covariates and their levels, so almost every treated unit ends up alone in its cell with no untreated partner. This is the curse of dimensionality applied to matching. ## The definition The propensity score of a unit is e(x) = P(T = 1 | X = x) the probability that a unit whose observed pre-treatment covariates equal `x` receives the treatment. Three things follow directly from that definition and are worth saying out loud in an interview: - It is a function of the covariates only. The outcome never appears in it, and neither does anything measured after treatment. - It is a probability between 0 and 1. In a fair randomised experiment `e(x) = 0.5` for everyone regardless of `x`; the score varies only because assignment was not random. - It is a nuisance quantity. Nobody wants to know the score. It exists to make the two groups comparable. ## The balancing property Rosenbaum and Rubin (1983) proved the result that makes the whole method work: the propensity score is a *balancing score*. Formally, the observed covariates are independent of treatment given the score, X ⊥ T | e(X) In words: pick any group of units that share the same propensity score, and within that group the distribution of the covariates is the same for the treated and the untreated. Whatever combination of age, tenure, size and history pushed a unit's score to 0.3, treated and untreated units sitting at 0.3 look alike on those covariates on average. That is a strong and slightly surprising claim, and it is the reason a one-dimensional match can reproduce what a twenty-dimensional match would have given. You are not throwing information away by collapsing the covariates into a scalar; for the purpose of balancing them, the scalar is sufficient. The second Rosenbaum–Rubin result carries this to causal identification. If treatment is unconfounded given the full covariate vector — potential outcomes are independent of treatment once you condition on `X` — then treatment is also unconfounded given `e(X)` alone. So under unconfoundedness plus positivity (every unit has a score strictly between 0 and 1, so both treatment states are possible for everyone), comparing outcomes between matched units identifies a causal effect. ## Where the theory stops and the work starts Three limits matter, and candidates who name them unprompted stand out. **The score is estimated.** The balancing theorem is about the true score, which nobody knows. You fit a model, get fitted values, and match on those. Nothing guarantees the estimated score balances anything, which is exactly why the matched sample must be checked covariate by covariate afterwards. Balance is an empirical claim about your data, not a theorem you get to cite. **Only measured, included covariates are balanced.** A covariate that was never recorded, or was recorded and left out of the score, is not balanced by any of this. Matching on observables handles confounding by observables and nothing more. The long-running LaLonde job-training literature is the standing illustration: non-experimental comparison pools drawn from national survey data produced estimates far from the experimental benchmark, and the Dehejia–Wahba reanalysis only approached it once rich pre-treatment earnings histories were included — with later work showing the result was sensitive to which sample and specification were chosen. **A highly predictive score is bad news, not good news.** People trained on prediction problems assume that a score which separates treated from untreated cleanly is a well-built score. It is the opposite. If most units have scores near 0 or near 1, the two populations barely overlap, few treated units have any plausible partner, and matching will either drop them or pair them with someone very different. The score's job is comparability, not accuracy. ## Common confusions The score is not the probability of a good outcome; it models assignment, not response. It is not a similarity metric between a unit and the treated group as a whole — two units can share a score for entirely different covariate reasons, which is why balance is checked in aggregate rather than pair by pair. And it must be built from pre-treatment variables only: anything measured after treatment can itself be affected by the treatment, so conditioning on it removes part of the very effect you are trying to estimate.