Your random-walk Metropolis sampler accepts 2% of proposals — what is wrong and how do you fix it?

## Reading the symptom A 2 percent acceptance rate in random-walk Metropolis has one dominant cause: the proposal distribution is much wider than the posterior. The candidate `theta' = theta + scale * noise` is being thrown far outside the region where likelihood times prior is appreciable, the density ratio is tiny, and the test rejects. Because a rejection re-records the current state, the output contains the same value repeated fifty times on average before anything changes. The chain is technically valid — it would eventually explore the posterior — but the compute is being spent on proposals that are thrown away. ## The mirror-image failure The instinct to push acceptance as high as possible is exactly wrong, and interviewers probe for it. Shrink the scale far enough and nearly every proposal is a hair away from the current point, the density ratio is close to one, and acceptance climbs towards 98 percent. Now nothing is rejected — and the chain still explores terribly, because every accepted move is microscopic. It takes an enormous number of iterations to cross the posterior, and consecutive draws are so similar that they carry almost no new information. So the two failure modes are: - **Scale too large:** acceptance near 2 percent, chain sticks at one point for long stretches. - **Scale too small:** acceptance near 98 percent, chain moves constantly but in useless increments. Both are the same disease measured differently: little effective distance travelled per unit of computation. The useful setting sits between them. ## The target acceptance rate The well-known asymptotic analysis of random-walk Metropolis on high-dimensional targets identifies an optimal acceptance rate of about 0.234, which is why 20 to 30 percent is the standard operating range for multi-parameter models. In a single dimension the corresponding optimum is closer to 0.44. These are guidelines derived under idealised assumptions, not laws; they are a good target to tune towards and a poor thing to defend to three decimal places. The right way to use them is as a cheap, always-available signal: an acceptance rate of 2 percent or 98 percent tells you something is wrong without any further diagnostics, while a rate in the twenties tells you the scale is not the limiting problem. ## How to tune The crude fix is manual: run a short pilot, look at the acceptance rate, multiply or divide the scale by a factor of two or three, repeat. It works and it is honest. The standard automatic fix is an adaptation phase. During a defined initial window, adjust the scale after each batch of iterations — increase it when acceptance is above target, decrease it when below — then freeze it and run the sampler you actually keep. The frozen-then-run structure matters: a chain whose proposal keeps changing based on its own history is no longer a plain Markov chain, and the usual guarantees do not directly apply. Two disciplined ways out are (a) adapt during a finite warm-up window whose draws are discarded, then hold the scale fixed, or (b) use a diminishing-adaptation scheme where the size of adjustments shrinks towards zero as the run proceeds. What you must not do is tune the scale using the whole run and then report those same draws as a clean sample. ## When a single scalar is the wrong knob One scale for all parameters assumes they all have similar posterior spread. Real models violate this constantly: an intercept on the scale of hundreds beside a coefficient on the scale of hundredths. One scalar is then simultaneously far too large for the tight directions, which drives rejections, and far too small for the wide ones, which drives crawling — and no single value fixes both. Tuning it will land you at some mediocre compromise with a passable acceptance rate and bad exploration. Two remedies: - **Rescale the parameters.** Standardise predictors and put the coefficients on comparable scales so one step size is meaningful. - **Use a proposal covariance.** Propose from a multivariate normal whose covariance is proportional to an estimate of the posterior covariance, obtained from a pilot run. This handles both differing scales and correlation between parameters, since the proposal is then stretched along the same directions the posterior is. Correlated parameters are the case where a scalar step is most obviously hopeless: the posterior is a narrow diagonal ridge, an isotropic proposal has to be small enough to fit the ridge's width, and it therefore takes forever to travel the ridge's length. ## What acceptance rate does not tell you An acceptance rate in the target range is necessary, not sufficient. It says the step size is sensible relative to local geometry; it says nothing about whether the chain has found all the posterior mass, whether it is stuck in one mode of a multimodal posterior, or whether the model is even identified. Treat it as the first, cheapest thing to check when a sampler behaves badly, and expect a follow-up question about what else you would look at before trusting the output.