Your triggered analysis shows +9% lift on the 3% of users who see the feature — what goes in the launch memo?

## Why one number is never enough A triggered result and a population result are both true, and they differ here by a factor of about thirty. A memo that quotes only the larger one is an overstatement of the launch's value by exactly that factor; a memo that quotes only the smaller one buries a feature that works. The honest artefact carries three things and labels them. ## The three figures 1. **The triggered effect with its interval.** Roughly +9% for users who opened the settings page. This is the efficacy number: does the design change behaviour for the people who meet it? 2. **The trigger rate.** 3% of assigned users. This is the reach number, and it is the bridge; without it neither of the other two can be derived from the other. 3. **The scaled population effect.** Roughly +0.3%, the figure that belongs next to other launches in a quarterly impact roll-up. ## Doing the scaling correctly On the absolute scale the arithmetic is exact under the assumption that untriggered users are unaffected: the population effect equals the trigger rate times the triggered effect. Convert the triggered lift into absolute units, multiply by 0.03, and convert back against the all-user baseline. The shortcut of multiplying the relative lift directly by the trigger rate is only correct when the control-arm metric level among triggered users is close to the level across all users. Users who reach a settings page frequently convert well above average, in which case the naive shortcut understates the population effect. Check that ratio before writing the number down, and say in the memo which convention was used. The interval scales with the estimate. The population figure inherits the triggered estimate's uncertainty, scaled by the same factor, plus whatever uncertainty sits in the trigger rate itself. A memo that reports the point estimate to two decimals and no interval is asserting a precision the experiment does not have. ## The assumptions worth writing down - **Untriggered users are unaffected.** Usually right for a self-contained surface, less obviously right when the change alters something shared, such as a ranking model or a notification budget. - **The trigger rate is treated as fixed.** It is a snapshot of today's discoverability, not a constant of nature. If the team later moves the entry point and exposure goes from 3% to 20%, the same triggered effect implies roughly +1.8% at the population level. - **The triggered effect persists.** A lift measured over two weeks on a surface people visit rarely may include novelty, and the memo should say what window it was measured over. - **Triggered users are not typical.** They are self-selected by having reached the surface. The effect estimate is valid for them; it is not a claim about what would happen if everyone were exposed. ## The strategic question the pattern raises This shape, a large effect on a small slice, is one of the most common findings in a mature product, and the interesting decision is not whether to ship. Shipping a positive, low-risk change with a small population impact is usually easy. The decision is where the next unit of engineering effort goes. A +9% effect on 3% of users caps the achievable population impact at whatever the reachable trigger rate is. If exposure could plausibly be lifted to 20% through placement or entry-point work, the ceiling moves to roughly +1.8%, six times the current value, without touching the feature itself. That comparison, effect work against reach work, is the argument the memo should set up, and the leader who frames it that way gets more out of the experiment than the one who reports a win and moves on. ## The organisational hazard Every incentive in a company pushes toward quoting the biggest defensible number. Triggered lifts are a reliable source of them, and they propagate: a +9% figure that leaves the memo without its trigger rate attached reappears in a deck as though it were a company-level result, and eventually somebody sums a year of such numbers and asks why revenue did not move. The defence is convention rather than vigilance. Make the readout template carry all three figures in fixed positions; make the roll-up accept only the population-scaled figure; require the trigger rate in any citation of a triggered effect. Those rules cost nothing to follow and remove the need to argue about it launch by launch. ## What good looks like A memo that opens with the population impact and its interval, states the trigger rate in the same sentence, gives the triggered effect as the efficacy evidence, lists the assumptions behind the scaling, and closes with the reach-versus-effect question for the next cycle. Nobody reading it can accidentally take away a number that is thirty times too large.