Can CUPED reduce variance in a test that runs only on new users with no pre-period history?

## Why the technique goes quiet The entire variance reduction of CUPED is `rho^2`, the squared correlation between the outcome and a covariate measured before the experiment. A population of brand-new users has no before. Every user's pre-period revenue, sessions or engagement is either missing or, if the pipeline imputes it, the same constant for everyone. A constant covariate has `var(X) = 0` and `cov(Y, X) = 0`. The optimal coefficient `theta = cov(Y, X) / var(X)` is `0/0` - undefined - and however you patch it numerically, the adjusted outcome equals the raw outcome up to a constant shift that cancels in a difference of means. The reduction is exactly zero. A pipeline that reports a gain here is measuring something else, and that is worth checking. ## What is still available The requirement is not that the covariate be a *pre-period metric*; it is that it be **determined before exposure**. On a new-user population several such things exist: - Acquisition channel or install source recorded at signup. - Device model, operating system, app version. - Country or locale. - Signup timestamp, day of week, hour. - Anything the user did between signing up and being exposed, **provided** exposure timing itself is not influenced by the treatment. If users are exposed on their first screen, this window is empty; if exposure happens at a later surface, there may be a real signal here, and the timing assumption needs checking rather than assuming. The honest expectation is modest. Per-user metrics are heavy-tailed and mostly driven by individual propensity, which coarse attributes capture badly. Where a user's own past revenue might correlate 0.6 or 0.7 with their future revenue, their acquisition channel is more likely to correlate 0.1 to 0.2 - a reduction of one to four percent. That is not a reason to build a pipeline; it may be a reason to use the attributes for balanced assignment instead. ## Mixed populations Most real experiments are not new-users-only; they are a mixture. Here CUPED still applies, and the practical handling is straightforward: impute the pooled mean of the covariate for users without history, so their adjustment term is zero and they neither gain nor are harmed. You can additionally include a has-history indicator so the two groups are not forced to share a single baseline. The realised reduction is then roughly proportional to the share of total metric variance contributed by users who do have history. If new users are a small tail of the population, the overall gain is close to what you would get on the returning-user population alone. If the experiment is dominated by newcomers, the headline gain collapses even though the code runs fine - which is why reporting a realised variance ratio per experiment beats quoting a single platform-wide number. ## What people reach for instead, and what to avoid Two moves are tempting and one of them is wrong. The wrong one is to fill the gap with early in-experiment behaviour - the first session after exposure, day-one activity, the first purchase. It is post-exposure, the treatment can move it, and the adjustment then removes part of the real effect while the confidence interval gets narrower. Narrow and wrong is the worst combination available. The defensible one is to move the variance reduction from the analysis to the design: balance the assignment across the pre-treatment attributes you do have - platform, country - so that at least the between-group component of the variance is not left to chance. The ceiling is the same as any covariate story - you can only remove variance that something you know explains - but it does not require a pre-period pipeline and it guarantees balance rather than correcting it afterwards. ## Choosing the pre-period when you do have one For populations that do have history, the covariate is not free of design choices. A very short pre-period makes `X` mostly measurement noise, which depresses the correlation. A very long one drifts away from current behaviour and excludes any user without full history, which can quietly change who is in your analysis. Something on the order of a couple of weeks before launch is the usual compromise, and the right way to pick it is to measure the realised correlation on historical data for each metric rather than to argue about it.