Why winsorise rather than trim extreme spenders when analysing an A/B test on revenue per user?

## The two operations Given a threshold `c` and a user's metric value `X`: - **Winsorising (capping)** replaces `X` with `min(X, c)`. The user stays in the analysis, contributing `c`. - **Trimming** removes the user from the analysis entirely when `X > c`. They look like variations of the same idea. In a randomised experiment they are not. ## Estimands Winsorising leaves you estimating the treatment effect on `E[min(X, c)]` — a well-defined quantity over the whole randomised population. Every user assigned is a user counted. Trimming leaves you estimating a difference between conditional means: `E[X | X <= c, treated] - E[X | X <= c, control]`. That conditions on the outcome, and the outcome is post-treatment. This is the classic post-treatment conditioning error. The treatment may itself determine who ends up above `c`, so the two surviving subpopulations are not the same kind of people any more, and the difference between them is not a causal effect on any pre-specified population. ## Why arm sizes matter Randomisation gives you two arms that are comparable in expectation. Trimming breaks the accounting: if the treatment makes users spend more, more treatment users cross `c` and are deleted, so the treatment arm loses proportionally more of its best outcomes. The observed difference can then be pushed toward zero or below it purely by the deletion rule — the very effect you are hunting is being selectively removed from one arm. A useful check when someone insists on trimming: report the number and proportion dropped in each arm. If those proportions differ by more than noise, the trim is itself reacting to the treatment and the comparison is contaminated. ## The standard-error trap Even outside a causal setting, a trimmed mean does not have standard error `s / sqrt(n_kept)` computed on the surviving values. The surviving sample has had its most variable members removed, so its sample standard deviation understates the variability of the estimator, and naive intervals come out too narrow. Correct inference for a trimmed mean uses a winsorised variance in the standard error rather than the trimmed sample's own variance. Winsorising avoids this entirely: you keep all `n` units and compute the usual standard error on the transformed values. ## When trimming is nevertheless defensible Trimming has honest uses in this context: - **Removing non-users.** If a value is impossible — a test account, a currency error, a duplicated charge — that is a data-quality fix, not tail control, and the unit should be excluded from both arms by a rule that has nothing to do with the treatment. - **Sensitivity reporting.** Publishing a trimmed comparison next to the winsorised primary, with the drop counts per arm, is a legitimate robustness exhibit. It just does not get to be the number the launch decision hangs on. ## Choosing between them in practice Default to winsorising for the decision metric because it preserves the intent-to-treat population, keeps the arms balanced by construction, gives you a clean estimand you can state in one sentence, and keeps standard-error computation ordinary. Reserve trimming for sensitivity, and if a trimmed and winsorised analysis disagree, that disagreement is information: it says the conclusion is being driven by users near or above the threshold, and that is a question about the feature, not about the estimator. ## The sentence that scores 'Winsorising transforms an outcome; trimming selects on an outcome. In a randomised experiment I will transform, not select, because selection on a post-treatment quantity breaks the comparability that randomisation bought me.'