In Lord's paradox, why do a gain-score analyst and a baseline-adjusted analyst disagree?

## The setup Lord described a university that records every student's weight in September and again the following June, for two groups of students who eat in different dining halls. Two statisticians analyse the same table. **Analyst 1** computes each student's weight change, June minus September, and averages it within each group. In Lord's telling, neither group's mean weight changed over the year: the average gain is zero for both. The conclusion is that there is no evidence of a differential effect of the dining halls on weight. **Analyst 2** models final weight as a function of initial weight and group membership — comparing students who arrived at the same September weight. On that comparison one group finishes systematically heavier than the other, and the conclusion is that there *is* a difference. Neither analyst has made an error. This is Lord's paradox. ## Why the two answers differ The groups differ at baseline: their September weight distributions are not the same. Every analysis of change has to supply, implicitly, an answer to the question *what would this student's June weight have been in the absence of a group difference?* The two analysts answer it differently. - **Change scores** assume that, absent an effect, a student's expected June weight equals their September weight. The baseline is the person themselves. - **Adjusting for baseline** assumes that, absent an effect, two students who arrived at the same September weight would have the same expected June weight regardless of group. The baseline is other people who started alike. Those are different assumptions about an unobserved counterfactual, and they are not nested, not reconcilable, and not checkable from the data. When the groups' baseline distributions overlap poorly, the two assumptions can and do disagree in sign. Nothing in the table distinguishes them, which is precisely why the paradox is instructive rather than merely annoying. ## The relationship to Simpson's paradox Same skeleton, continuous covariate. In Simpson's paradox a comparison reverses depending on whether you condition on a categorical variable; in Lord's paradox it reverses depending on whether you condition on a continuous baseline measurement. In both cases the arithmetic is unambiguous and the disagreement lives entirely in the choice of what to hold fixed — a choice that expresses a belief about how the data were generated. ## The extra difficulty: the grouping was never assigned There is a second problem in Lord's example that a lead should name explicitly. The groups were not created by an intervention anyone controlled. Asking what a student's weight *would have been* in the other group requires imagining a manipulation that was never defined — and when the grouping is an attribute rather than an action, the counterfactual may not be well defined at all. Before arguing about which analysis is better, a principal-level answer asks what intervention the question refers to. Many "paradoxes" of this shape dissolve once someone insists on stating the intervention. ## When the two analyses agree If group membership had been randomly assigned, baseline weight would be balanced between the groups in expectation. Both analyses then target the same quantity and differ only in precision, with the baseline-adjusted version typically the more precise because it removes variance explained by the starting value. The disagreement is a symptom of baseline imbalance in a non-randomised comparison, not a defect of either method. That is worth saying out loud, because it locates the fix in the design rather than in the modelling. ## What a lead actually does about it 1. **Fix the estimand before the analysis.** Write down, in words, the quantity of interest and the intervention it refers to, and get agreement on it while the result is still unknown. 2. **State the identifying assumption in plain language.** "We assume two students who arrived at the same weight would have ended at the same weight had they eaten in the same hall" is a sentence stakeholders can argue with. A model specification is not. 3. **Run both and treat the gap as information.** If change scores and baseline adjustment disagree materially, that is a finding about baseline imbalance, and it belongs in the write-up rather than being resolved silently by whoever holds the keyboard. 4. **Do not let the result choose the method.** Pre-specify. The single most damaging version of this paradox is an analyst who tries both and reports the one matching the expected story. 5. **Push the fix upstream.** Where the comparison matters and assignment is feasible, randomise or otherwise balance the baseline; where it is not, be explicit that the conclusion rests on an assumption the data cannot test. ## The signal in an interview Weak answers pick a side — "you should always adjust for baseline" or "change scores are unbiased". Strong answers say that both estimates are correct arithmetic, name the counterfactual each assumes, observe that baseline imbalance in a non-assigned grouping is what makes them diverge, and describe the process by which the choice gets made and recorded before anyone sees the result.