What does the Gelman-Rubin R-hat statistic compare across MCMC chains?

## What R-hat is for Running one MCMC chain gives you no external reference: whatever the chain does looks, from the inside, like the posterior. The Gelman-Rubin diagnostic buys a reference by running **several chains from deliberately dispersed initial values** and asking a simple question — do these independent explorations agree with one another? If they have all converged to the same target, then the spread of one chain's draws should already tell you about the spread of the pooled draws. If they are still in different places, pooling them inflates the apparent variance, because you have added the differences *between* chains on top of the variation *within* each one. ## The construction For one scalar parameter, with `m` chains of `n` retained draws each: - `W` = the average of the within-chain variances. This *under*-estimates the posterior variance while the chains are still confined to their own neighbourhoods, because no single chain has yet toured the whole distribution. - `B` = the variance of the chain means, scaled by `n`. This captures how far apart the chains sit. - `V_hat = ((n - 1) / n) * W + (1 / n) * B`, a pooled variance estimate that *over*-estimates the posterior variance while the chains disagree. `R_hat = sqrt(V_hat / W)`. Read it as the factor by which your interval estimates might still shrink if you kept sampling — hence the original name, potential scale reduction factor. As the chains mix, `B` shrinks toward the within-chain scale, the numerator and denominator converge, and R-hat descends to 1 from above. ## Thresholds and the split refinement The historical rule of thumb was `R_hat < 1.1`. Contemporary guidance is much stricter — around **1.01** — because the older threshold lets through chains whose effective sample size is far too small for reliable tail summaries. Interviewers do not usually care which number you quote as long as you say the modern bar is tight and that the direction is "close to 1 from above". Plain R-hat has a hole: a single chain that drifts slowly in one direction has a large within-chain variance, which flatters the denominator. **Split R-hat** closes it by cutting each chain into halves and treating the halves as separate chains. A drifting chain then disagrees with itself, and the statistic rises. Rank-normalised variants further protect against heavy tails and infinite-variance parameters, where the raw variance ratio is not well behaved. ## The failure it is designed to catch The canonical picture: four chains launched from dispersed starting points, three of them settling into the same region and one parking somewhere else entirely and never leaving. The chain means are far apart, `B` is large relative to `W`, and R-hat comes back at something like 1.4. The correct read is not "run longer and it will average out" — it is that the sampler is not moving between regions of the posterior, so no summary computed from the pooled draws means anything. The remedies are structural: reparameterise, tighten the model, or accept that the posterior is genuinely multimodal and report it as such rather than pooling. ## Necessary, not sufficient R-hat near 1 says the chains agree. It does not say they are right. - If all chains are initialised in the same small region, they can agree with each other while jointly missing posterior mass elsewhere. Dispersed initialisation is part of the diagnostic, not an optional extra. - R-hat near 1 with a tiny effective sample size means the chains agree about a distribution they have each sampled only a handful of independent times. R-hat and effective sample size are checked together; neither substitutes for the other. - A sampler flagging pathologies during its trajectories can produce a comfortable R-hat while systematically under-visiting a hard region of parameter space. ## Practical protocol Run at least four chains from dispersed inits. Discard warm-up. Compute split R-hat per parameter — including derived quantities and the log posterior density, not just the parameters you care about, since a problem often shows up first in a scale parameter. Look at the *maximum* R-hat over parameters, not the average. Confirm with a trace plot: converged chains overlay each other as one fat band, while chains at R-hat 1.4 show visibly separated ribbons. If any parameter fails, treat the entire fit as unusable rather than reporting the parameters that happened to pass.