How do you choose the segment partition when forecasting an aggregate rate as a weighted sum?

## The identity you are relying on Decomposing a metric is an application of the law of total probability: ``` P(convert) = sum over segments s of P(convert | s) * P(s) ``` In forecasting language: **the aggregate rate is the mix-weighted average of the segment rates**. Forecasting it therefore splits into two separate forecasting jobs — one for the conditional rates, one for the mix — and the value of the whole exercise depends on those two being easier to predict than the aggregate itself. ## Hard requirements on the partition **Mutually exclusive.** If a unit can fall into two segments, its contribution is counted twice and the weighted sum exceeds the truth. Overlapping labels (channel *and* device used side by side as if they were one partition) are the most common version of this error; the fix is to use the cross-product as a single partition, not two lists. **Exhaustive.** Every unit must land somewhere, including the awkward residual traffic. If the shares sum to less than 1, the forecast is biased low by whatever was dropped. Keep an explicit "other" bucket rather than silently discarding it. **Knowable in advance.** The segment label must be assignable from information available before the outcome. Segmenting by "users who converted versus users who did not" produces a decomposition that fits history perfectly and forecasts nothing. Anything measured downstream of the outcome, or contaminated by it, has the same defect in subtler form. **Stable definitions.** If the segment definition itself changes between periods — a taxonomy revision, a new channel folded into an old one — the historical rates and shares are no longer comparable, and the decomposition silently breaks. ## The property that makes a partition *useful* Beyond legality, the partition earns its keep when the conditional rates are **more stable than the aggregate**. If each segment's rate is roughly flat while the mix moves around, the aggregate is volatile purely through mix, and the decomposition explains and predicts that volatility. If instead every segment's rate swings as much as the aggregate, you have replaced one hard forecasting problem with several, plus a mix forecast, and gained nothing. This also gives you the diagnostic that most often justifies the work in the first place: **the aggregate can move even though no segment rate moved at all**. A shift of traffic toward a lower-converting segment drags the overall rate down while every conditional rate is flat or rising. Without the decomposition that pattern is invisible and gets misdiagnosed as a product regression. ## The granularity tradeoff Refining a partition reduces within-segment heterogeneity — conditional rates become more meaningful — but each cell is estimated from less data, so both the rate and the share estimates become noisier, and the noise propagates into the weighted sum. Practical guardrails: - Split only where the conditional rates are genuinely different; a split that produces near-identical rates adds variance and no signal. - Keep a minimum volume per cell, and pool or shrink thin cells toward the aggregate rather than trusting a rate estimated from a handful of events. - Prefer a small number of stable, business-meaningful dimensions to a deep cross-product with mostly empty cells. - Remember that the mix itself must be forecastable; a segmentation defined on something that moves erratically for exogenous reasons pushes the difficulty into the weights. ## Validating the choice The decomposition is a modelling claim, so test it. Backtest the decomposed forecast against a pooled forecast on held-out periods and compare errors. Check whether conditional rates really were more stable historically than the aggregate. Track whether the residual — the gap between the reconstructed and the actual aggregate — is close to zero; a persistent gap usually means the partition is not exhaustive or the shares are measured on a different population from the rates. ## Organisational considerations A partition also becomes a reporting contract. Once teams are held to segment-level numbers, the segmentation is expensive to change, and every future analysis inherits it. That argues for choosing dimensions that are meaningful to the business and stable over years, not the split that happens to minimise this quarter's forecast error. It also argues for being explicit that a single blended rate is a mix-weighted average, so that nobody reads a mix-driven move as a performance change. ## A short checklist 1. Disjoint, exhaustive, shares summing to 1. 2. Assignable strictly before the outcome. 3. Conditional rates more stable than the aggregate. 4. Enough volume per cell to estimate both the rate and the share. 5. Mix itself forecastable. 6. Definitions stable across periods. 7. Backtested against the pooled alternative.