What changes when an active-user metric uses a 28-day activity window instead of 7?

## What an activity window is doing An activity-window metric is a binary per-user indicator with a lookback attached: a user counts as active on a given day if they performed at least one qualifying action at some point in the preceding N days. Three choices define it completely — the qualifying action, the window length N, and the reference date the window looks back from. Changing any one of them changes the number, so all three belong in the metric definition. ## Longer versus shorter **Level.** A longer window can only include more users, so a 28-day count is always at least as large as a 7-day count on the same action and the same date. The two are not different measurements of the same quantity; they are different quantities, and comparing a 28-day figure against a 7-day figure is a category error. **Responsiveness and memory.** The window is a memory. With N = 28, a user who stops today continues to count for the next 27 days, so a genuine collapse in activity decays out of the metric gradually rather than appearing at once. That smoothing is useful for reporting a stable trend and dangerous for detecting a regression quickly. With N = 7, a stop shows up within a week, which is what you want for an incident or a fast experiment, at the cost of a much bouncier series. **Seasonality.** Weekly rhythm is the dominant cycle in most consumer and business products. A window that is a whole number of weeks contains the same number of each weekday, so weekday composition stays constant as the window slides. This is the practical reason 28 days is preferred over 30: 28 is exactly four weeks, whereas a 30- or 31-day window keeps changing which weekdays it double-counts, producing a sawtooth that is easy to misread as a trend. A 7-day window shares the whole-week property but has far less averaging behind it. **Sensitivity to a change.** A long window sets a low bar — one action in four weeks — so it saturates. Many products have most of their base clearing that bar, which leaves little headroom for a treatment to move the number and makes the metric insensitive as an experiment readout. A shorter window sets a higher bar and typically discriminates better between arms, which is why experiment readouts often use tighter windows than executive reporting does. ## The experiment-duration problem This is the part candidates most often miss. To claim a treatment caused a change in a 28-day active metric, the 28 days being examined must all lie after exposure. If the experiment runs for 14 days and you compute a 28-day window at the end, roughly half of each user's window predates their exposure. Activity in that pre-exposure half is identical in expectation across arms, so it adds signal-free mass to both, and any real effect is diluted toward zero. A null result under those conditions tells you almost nothing. The options are: - Run long enough that every enrolled user has a full window observed after exposure — for a 28-day window, that is 28 days after the last enrolment, not after the first. - Use a shorter window that fits the test, and state that the readout is a 7-day or 14-day activity metric rather than the reporting metric the company usually quotes. - Read the window-based metric as a directional secondary and decide on a metric that is defined within the test's horizon. A related trap is left-censoring at the start of a product's life or a new cohort's life: users who have not existed for 28 days cannot have a full window, and including them mechanically depresses the metric. ## Getting the definition right Write down: the qualifying action (opened the app? completed a meaningful action? passive background sync definitely should not count); the window length and its justification; whether the window rolls daily or snaps to calendar boundaries; and how a user with activity exactly on the boundary day is treated. Then leave it alone. Because the boundary is arbitrary, changing N after seeing results is one of the easiest ways to manufacture a favourable number, and it is visible to anyone who reads the history of the definition. Pre-register the window with the experiment, and if you genuinely need both a fast and a stable read, publish both, permanently labelled, rather than switching between them.