How do O'Brien-Fleming and Pocock group-sequential boundaries differ?

## The same guarantee, two shapes Both families are group-sequential boundaries: a table of critical values, one per planned analysis, constructed so that the probability of crossing at **any** of them under the null equals the nominal level. What separates them is how the error budget is distributed across time, which shows up as the shape of the boundary on the standardised scale. ## Pocock: a flat boundary Pocock's boundary uses the **same** critical value `c` at every look, with `c` solved so that the overall crossing probability equals the nominal level. It grows with the number of looks: roughly 2.18 for two looks, 2.29 for three, 2.36 for four, and 2.41 for five, at two-sided 0.05. Read as a per-look nominal level, 2.36 corresponds to about 0.018 rather than 0.05 — noticeably stricter than a one-shot test, but reachable. Consequences: - Early stopping is a genuine possibility. A moderate true effect has real probability of pushing the statistic past 2.36 at the first or second look. - The final analysis is penalised for the whole run. If the effect is real but small, so that the statistic only reaches, say, 2.1 at full data, the experiment is declared inconclusive even though a fixed-horizon test at 1.96 would have rejected. - Holding power fixed, the design needs a meaningfully larger maximum sample than a fixed-horizon design — on the order of 15% to 25% more with several looks. ## O'Brien-Fleming: stringent early, near-normal at the end The O'Brien-Fleming boundary makes the critical value a decreasing function of the information fraction: with `K` equally spaced looks the cutoff at look `k` scales roughly as `c * sqrt(K / k)`. With four looks at two-sided 0.05 the sequence runs approximately 4.05, 2.86, 2.34, 2.02. Consequences: - The first look is effectively unreachable. Only a spectacular effect — or a catastrophic regression, in the harm direction — will cross it, which is exactly the intent: early data are noisy, and an early stop on noise is expensive. - Almost none of the budget is consumed early, so the final cutoff, about 2.02, is barely stricter than the fixed-sample 1.96. Power at full data is close to that of a fixed design, and the maximum sample inflation is a few percent. - Early stopping does happen, but essentially only for large effects, which is the case where stopping early is most obviously right. ## Choosing between them The question to ask is what an early stop is worth relative to a missed small effect. - Choose an O'Brien-Fleming shape when the endpoint decision matters most and you mainly want an escape hatch for enormous wins or losses. This is the usual default in product experimentation, where small effects are common and abandoning them because the boundary was flat is costly. - Choose a Pocock shape when the cost of running is high, effects are expected to be large if they exist, and finishing weeks earlier is worth losing sensitivity at the end. - Anything in between is available: a power-family spending function with a tunable exponent interpolates between the two shapes, so you are not restricted to the two classical tables. ## Details that separate a good answer - Both boundaries have a harm-direction counterpart. A two-sided boundary can be crossed downward, which is how a sequential design stops a damaging change early. - The boundaries are computed for the **standardised** statistic under a known correlation structure between successive looks. If the actual information fractions drift from the plan — traffic arrives faster or slower than expected — a spending-function implementation recomputes the boundary at the realised fractions rather than reusing the original table. - Under either boundary, the point estimate at the moment of stopping is biased away from the null, because crossing selects for a favourable draw. A design that stops early should report a bias-adjusted estimate alongside the decision, and the earlier the stop, the larger the correction. - Neither shape is more or less honest than the other. They control exactly the same overall error rate; they trade when that error may be spent. ## What interviewers listen for The strongest answers give the shape in one sentence each, quote the direction of the tradeoff correctly — Pocock stops earlier and pays at the end, O'Brien-Fleming almost never stops early and pays almost nothing at the end — and then tie the choice to the economics of the experiment rather than declaring one boundary universally superior.