Why does the assignment mechanism, how units came to be treated, determine what effect you can estimate?

## Definition The **assignment mechanism** is the process that generated the treatment indicator `D` for each unit — in probability terms, the distribution of `D` given the units' covariates and their potential outcomes `Y(1)` and `Y(0)`. It is the second half of the potential outcomes framework, and the half that is easy to skip. The first half says each unit has two outcomes and you see one; the second says which one you see is decided by a process, and that process is the only thing standing between the data and a causal claim. The useful mental test is one question: **could the assignment have depended on the potential outcomes themselves?** Everything else follows from the answer. ## Three mechanisms, one treatment Take a single treatment — a premium onboarding tour for new accounts — and vary only how accounts came to receive it. ### Randomised assignment A coin flip, or a fixed probability applied to everyone. `D` is generated by a device that has no access to the outcomes, so the treated and untreated groups differ only by chance. The average outcome of the untreated group is a fair stand-in for the average outcome the treated group would have shown untreated, so the difference in means estimates the average effect. Under this mechanism the effect on the treated and the effect on the untreated coincide in expectation with the population average effect, so the distinction between those estimands stops mattering — a luxury that disappears the moment assignment is not random. ### Self-selection Accounts opt in to the tour. Now `D` is generated by the account's own motivation, product intent and free time — traits that also drive retention and expansion. The treated group would have looked better than the untreated group with no tour at all, so the observed gap contains the effect plus that pre-existing difference. Worse, the driving trait is often unmeasured: you can adjust for plan tier and company size and still be adjusting for the wrong thing. Self-selection is the most common mechanism in product data and the one most often analysed as if it were the first case. ### A deterministic rule Operations decides every account with an activation score of 80 or above gets the tour, no exceptions. Assignment is now a function of an observed variable, which sounds like an improvement — nothing unmeasured is driving it. But the rule leaves no variation to exploit over most of the range: at a score of 90 every account is treated, so there is no comparable untreated account at that score, and at 40 the reverse. The comparison you would want simply does not exist in the data except close to the threshold, where accounts on either side are plausibly similar. Any estimate that spans the whole range rests on a model extrapolating outcomes into regions where one arm is never seen, and that model, not the data, is doing the work. ## Why this decides the estimand, not just the method Each mechanism supports a different claim: - Randomisation across the whole population supports a population-average claim. - Randomisation inside a segment supports a claim about that segment only. - Self-selection supports nothing without an argument that the selection is fully captured by what you measured. - A deterministic rule supports a local claim around its threshold much more comfortably than a global one. This is why experienced practitioners ask "how did these units come to be treated?" before asking what the data looks like. The answer bounds what any analysis can deliver, and no estimator raises that ceiling. ## Interview notes When given a dataset and asked for the effect of something, the first question back should be about the mechanism. If the interviewer says the treatment was rolled out to whoever asked for it, say so plainly and describe what would have to be true for an adjusted comparison to be credible. Candidates who jump straight to a model are answering the second question before the first. ## Traps - Assuming a mechanism is 'as good as random' because nothing obviously biased is happening; absence of a story is not evidence of randomness. - Believing that a rule based on observed data is automatically safe, when it can eliminate the comparison entirely. - Describing the mechanism vaguely: 'the team picked some accounts' is not a mechanism, and the details usually decide the answer.