Propensity matching dropped your enterprise accounts for lack of common support - what do you report?

## What common support means Common support, or overlap, is the region of the propensity score range where both treated and untreated units actually exist. It is the practical face of the positivity condition that any adjustment method needs: for the comparison to mean anything, a unit like this one must have been able to end up in either arm. When enterprise accounts have estimated scores clustered near 0.99, that condition has failed for them empirically. Essentially every account of that kind was treated. There is no untreated enterprise account to serve as a counterfactual, and no amount of statistical machinery conjures one. Matching therefore leaves those treated units unmatched, and they fall out of the analysis. ## The estimand changes, and it changes quietly This is the part that gets missed. Before trimming, the target was the average effect on the treated: every treated account, enterprise included. After trimming, the number you can defend is the average effect on the treated units that lie inside the region of overlap. Those are different quantities about different populations, and only the second one is supported by the data. Nothing in the pipeline announces the switch. The estimate comes out with a tidy confidence interval, the balance table looks excellent — often *better* than before, because the hardest units were removed — and the write-up says "effect of the programme on treated accounts". That sentence is now false, and nobody reading the report can tell. So the first duty is naming. The results section should say which population the number describes, in the language the business uses: "among mid-market accounts with a comparable untreated counterpart" rather than "among treated accounts". ## Quantify the loss in the units the audience cares about Counting discarded units is necessary but not sufficient, because accounts are not interchangeable. Three numbers belong in the report: 1. **How many treated units were discarded**, as a count and a share of the treated group. 2. **How they differ** from the retained units on the covariates that drive the outcome — size, tenure, region, engagement. 3. **What share of the outcome they carry.** Losing 8% of accounts sounds negligible until you note that those accounts represent 60% of revenue. Then the study covers a minority of what the decision is about. That third number converts a methodological footnote into a business fact, and it is what separates a principal-level answer from a competent one. ## The judgment call With those numbers in hand, there are three honest paths. **Report the trimmed estimate, clearly labelled.** Correct when the retained population is one somebody will act on. If the rollout decision concerns mid-market accounts, an effect estimated on comparable mid-market accounts is exactly the right deliverable. **Declare the question unanswerable with this data, and say what would answer it.** Correct when the discarded segment *is* the question. A staged or randomised rollout within the enterprise segment creates the variation that history did not; so does finding a comparison pool that actually contains untreated enterprise accounts, such as another market or an earlier period before the programme reached that tier. Saying this early is far cheaper than saying it after three weeks of analysis. **Split the report.** Estimate where support exists, and state explicitly that the enterprise segment is out of scope with a named plan to cover it. This is usually the version stakeholders can act on. ## What not to do The tempting move is to relax the matching until nothing is discarded — widen the distance allowed, or fall back to a model that extrapolates a prediction into a region where no untreated units were ever observed. Both replace a visible, countable restriction with an invisible assumption. A trimmed sample tells you exactly what you do not know and how much of it there is. A forced match, or a model asked to predict outside the range of its data, produces a number with no warning label attached, and its error does not shrink as the sample grows. The second thing not to do is decide the trimming rule after seeing the outcomes. The overlap rule belongs in the analysis plan: how the region is defined, what happens to units outside it, and what fraction of treated units you are prepared to lose before the study is declared infeasible. Fixed in advance, it is a design decision. Chosen afterwards, it is a knob that gets turned until the estimate looks acceptable. ## The organisational point Teams that run many observational studies benefit from making this a standing requirement rather than a judgment each analyst re-makes: every causal write-up names its population, reports discarded units with their share of the outcome, and states the overlap rule that was fixed before outcomes were examined. That single convention prevents most of the quiet estimand drift that makes observational results untrustworthy over time.