Why are the warm-up (burn-in) draws of an MCMC chain discarded before summarising the posterior?

## The problem warm-up solves Markov chain Monte Carlo does not draw independent samples from the posterior. It constructs a chain of dependent states whose *stationary* (long-run) distribution is the posterior. The theory guarantees that if you run the chain long enough, the distribution of its state converges to the posterior no matter where you started. It says nothing reassuring about iteration 3. You have to start the chain somewhere — a random point, a crude initial guess, all parameters at zero. That point is almost always far out in the tails, in a region the posterior gives very little mass to. The chain then drifts from there toward the **typical set**: the region that actually holds nearly all posterior probability. Draws collected during that drift are samples of "where a chain that started at my arbitrary initial value is after k steps", not samples of the posterior. Averaging them into your posterior mean drags the estimate toward the initial value. That is *bias*, and unlike Monte Carlo noise it does not shrink just because you collect more draws afterwards — it shrinks only because you throw those early draws away. ## The second, sharper reason: adaptation Modern samplers spend the warm-up period tuning themselves: choosing a step size to hit a target acceptance rate, estimating a mass matrix or covariance for the proposal geometry. During that period the transition rule itself is changing from iteration to iteration. A chain with a changing kernel is not a homogeneous Markov chain targeting the posterior, so the usual invariance argument does not apply to those iterations at all. Even if the trace *looks* flat and well-behaved during adaptation, those draws are not posterior samples and must be discarded. This is why the modern name is "warm-up" rather than "burn-in": the phase is doing tuning work, not merely forgetting the start. ## How much to discard There is no formula. Practical guidance: - A common default is to discard the first half of every run, then use the second half for inference. It is wasteful but safe. - Judge it with the diagnostics you would run anyway: start several chains from *dispersed* initial values and check that, after the discarded portion, they overlap on the trace plot and their between-chain and within-chain variation agree. - Compute convergence statistics on the *post*-warm-up draws only. Computing them on the full run mixes the transient into the diagnostic and can either hide a problem or invent one. ## What warm-up does not do Warm-up is a fix for one specific failure: dependence on the starting point. It does not: - **Reduce autocorrelation.** Successive draws remain correlated after warm-up; that is a separate property, measured by effective sample size, and no amount of discarding at the front changes the correlation structure of what remains. - **Rescue a chain stuck in the wrong place.** If the chain has entered a region it cannot leave, throwing away the first half just gives you a longer run of the same stuck behaviour. The trace will look deceptively calm. - **Fix a misspecified or badly conditioned model.** A chain that will not settle after a long warm-up is usually telling you about the geometry of the posterior, not asking for more iterations. ## The cost side Discarding is not free. Every discarded draw is a draw you paid for and cannot use, so the remaining sample is smaller and the Monte Carlo error of your posterior summaries is larger. The trade is bias against precision: too little warm-up leaves initialisation bias in the estimate; too much leaves an honest but noisier estimate. Because bias is the more dangerous failure — it does not announce itself in an interval — the conventional choice errs toward discarding generously, and then buys the precision back by running the sampler longer rather than by keeping the transient. ## What an interviewer is listening for That you can say *why* the early draws are not posterior draws (start dependence, plus a changing kernel during adaptation), that you check the decision with dispersed chains rather than a rule of thumb, and that you do not oversell warm-up as a cure for mixing or for a chain trapped away from the bulk of the posterior.