Why did observational studies of hormone replacement therapy overstate its heart benefit?

## What happened For years, observational cohorts consistently found that women taking hormone replacement therapy had lower rates of coronary heart disease than women who did not. The finding was reproduced across studies, was biologically plausible, and shaped prescribing. When the question was finally settled by a large randomized trial, the cardioprotective effect was not there: the trial found no coronary benefit, and the combined estrogen-progestin arm was stopped early because the overall risk profile was unfavourable. This is the most instructive case in applied causal inference precisely because the observational studies were not sloppy. They were large, prospective, and adjusted for a long list of covariates. What failed was the identification assumption underneath the adjustment. ## The mechanism: healthy-user and healthy-adherer effects Treatment was not assigned; it was chosen, jointly by patients and their doctors. The women who chose it differed systematically: - **Healthy-user effect.** People who take a preventive medication are, on average, people who do other health-protective things: they exercise, they attend screenings, they have better access to care, they have more money and more education. - **Healthy-adherer effect.** Among those prescribed something, those who keep taking it are a further-selected group. Randomized trials have found that participants adhering to *placebo* have better outcomes than those who do not, which is unambiguous evidence that adherence marks something about the person rather than something about the pill. - **Prevalent-user selection.** A cohort that recruits women already on therapy has, by construction, excluded anyone who started and stopped because of an early adverse event, enriching the treated group with people who tolerated it well. Each of these makes the treated group look better regardless of the drug, which is exactly what conditional ignorability, `(Y(1), Y(0)) independent of T | X`, forbids. ## Why the adjustment did not save it A covariate set can be long and still be the wrong set. Age, smoking, blood pressure and body mass are outcomes and correlates of health-seeking behaviour, not measurements of it. Adjusting for them removes the part of the confounding they capture and leaves the rest, and residual confounding after adjustment looks identical, in the output, to a real effect. There is no statistic in the analysis that flags the problem, because the assumption concerns something absent from the data. Two warning signs were available in principle. First, the estimated effect implied benefit across many outcomes with no shared mechanism, which is the fingerprint of a generally healthier treated group rather than of a drug. Second, the treatment decision was known to be socioeconomically patterned, so the mechanism argument for exchangeability was weak from the start. ## What a strong candidate does with this - **Starts from the decision.** Who chooses this treatment, on what information, and does that information predict the outcome? If the answer is a disposition rather than a recorded variable, say so up front. - **Asks whether the covariates measure the driver or merely correlate with it.** A proxy measured with error leaves residual confounding proportional to the error. - **Prefers designs where assignment does not depend on the disposition.** An experiment where feasible; failing that, a source of variation in treatment that is plausibly unrelated to health-seeking behaviour. - **Restricts to new users.** Comparing people from the moment they start therapy avoids the survivor enrichment of a prevalent-user cohort. - **Names the direction of the residual bias.** Healthy-user selection biases a preventive treatment toward looking beneficial, so an observed protective effect should be discounted rather than taken at face value. - **Adjusts the language, not just the model.** If the assumption cannot be defended, report an association and say what would be needed to make it causal. ## The general lesson The failure was not statistical noise, and no sample size would have fixed it, because bias does not shrink with n. It was a failure of the identification assumption: the covariate set did not contain the driver of treatment choice. When an observational result and a randomized result disagree, the default explanation is that the observational study's exchangeability assumption was wrong, not that the trial was unrepresentative, though population differences are worth checking as a secondary question.