How does a hierarchical prior partially pool eight noisy per-group effect estimates?

## The problem partial pooling solves You have eight groups — eight sites, eight schools, eight cohorts — and one effect estimate per group, each with its own standard error. Two obvious analyses are available and both are bad. **No pooling** takes each group's estimate at face value. With small per-group samples this hands you eight noisy numbers, ranks them, and invites everyone to celebrate the top group and investigate the bottom one — most of which is sampling noise. Extremes are the most likely to be extreme by luck, so the ranking will not replicate. **Complete pooling** ignores the grouping and reports one overall number. This is stable but denies real differences and is useless when the question is precisely which groups differ. A hierarchical prior is the third option, and it is a *prior*, not a trick: it states that the eight true effects are related to one another. ## The structure The standard version has two stages: 1. **Observation stage.** Group j's estimate `y_j` is centred on that group's true effect `theta_j` with a standard error `s_j` you already know from the group's own sample size. 2. **Group-level prior.** The eight true effects are themselves drawn from a common distribution: `theta_j` is normal with mean `mu` and standard deviation `tau`. The second stage is the prior on the group effects, and its parameters `mu` and `tau` are unknown, so they get their own (usually weakly informative) priors and are estimated from the data. That is what makes the pooling adaptive rather than a knob you set by hand. ## What comes out Conditional on `mu` and `tau`, the posterior mean for group j is a precision-weighted average: `theta_j_hat = (tau^2 * y_j + s_j^2 * mu) / (tau^2 + s_j^2)` Read the weights. The group keeps a share `tau^2 / (tau^2 + s_j^2)` of its own estimate and borrows the rest from the common mean. Three consequences follow immediately: - A group with a **large standard error** — small sample, noisy measurement — has a big `s_j^2`, so it shrinks hard toward `mu`. - A group with a **small standard error** keeps almost all of its own estimate. - The **amount of shrinkage is not chosen by you**; it follows from how much real variation between groups the data support. If the spread of the eight estimates is no larger than their standard errors alone would produce, `tau` is estimated near zero and the model pools almost completely. The classic eight-groups demonstration is exactly this: eight programme effects, several of them apparently large, with standard errors big enough that the estimated between-group spread is small — so the partially pooled estimates all sit close to the overall mean, and the apparently outstanding group loses most of its lead. The honest reading is that the raw ranking was mostly noise. ## Why this predicts better Efron and Morris's baseball demonstration made the point empirically: take each player's batting average over his first 45 at-bats, shrink it toward the group mean, and the shrunken values predict the rest of the season better than the raw early averages do — for nearly every player. Shrinkage trades a little bias for a large reduction in variance, and on out-of-sample prediction that trade wins. The Bayesian hierarchy is the version where the amount of shrinkage is inferred rather than imposed. ## Where it goes wrong **Too few groups.** With three to five groups, `tau` is barely identified: the data cannot distinguish "no real variation" from "moderate variation drowned in noise", and whatever prior you put on `tau` ends up driving the pooling. Use an informative prior on `tau` and say so, or accept that you are close to complete pooling. **A careless prior on the spread.** A flat or improper prior on `tau` is a known hazard — it can leave the posterior improper or push the fit toward implausibly large between-group variation. A half-normal or other positive, weakly informative prior on `tau`, scaled to what a plausible between-group difference looks like in the units you care about, is the standard fix. **Shrinking things that are not exchangeable.** The group-level prior asserts that, before seeing the data, the groups are interchangeable. If one group is a different product, a different population or a different measurement protocol, pooling it with the rest borrows strength from the wrong place. The repair is to model the difference — add the covariate that explains it — not to abandon the hierarchy. **Reading shrunken estimates as if they were raw.** Shrunken group means are deliberately biased toward the centre. They are the right thing to act on for decisions about individual groups, but they are not the right input to a second analysis that assumes unbiased inputs. ## What an interviewer is listening for The weight formula, or at least the direction of it — noisier groups move more — plus the recognition that `tau` is learned rather than set, plus one failure mode you have actually hit. Candidates who describe partial pooling as "averaging the groups a bit" have not understood that the amount of averaging is the inference.