In Metropolis-Hastings, why does the unknown normalising constant cancel out of the acceptance ratio?

## The rule Metropolis-Hastings builds a chain over parameter space from two ingredients: a proposal distribution and an acceptance test. Starting at the current value `theta`, one iteration is: 1. Draw a candidate `theta'` from a proposal density `q(theta' | theta)`. 2. Compute the acceptance probability ``` alpha = min(1, [ p(theta') * q(theta | theta') ] / [ p(theta) * q(theta' | theta) ]) ``` 3. Draw `u` uniform on (0, 1). If `u < alpha`, the next state is `theta'`; otherwise the next state is `theta` again. Step 3 matters more than beginners expect. A rejection is not a skipped iteration — the current value is written into the output a second time. A chain that rejects heavily therefore contains long runs of repeated identical values. ## Where the constant goes The target `p` in that formula is the distribution you want to sample, here the posterior: ``` p(theta) = p_unnorm(theta) / Z, p_unnorm(theta) = p(y | theta) * p(theta), Z = integral of p_unnorm ``` Substitute this into the ratio: ``` p(theta') / p(theta) = [ p_unnorm(theta') / Z ] / [ p_unnorm(theta) / Z ] = p_unnorm(theta') / p_unnorm(theta) ``` The `Z` divides out exactly, because it is the same number in both places — it does not depend on `theta`. So the sampler only ever needs to evaluate likelihood times prior at two points and compare them. This is the reason Metropolis-Hastings is usable at all on real posteriors: the evidence integral that makes the posterior intractable in closed form never appears in the algorithm. The same cancellation happens with any target known up to a constant, which is why the method also shows up outside Bayesian inference wherever an unnormalised score function is available. ## The proposal correction term The `q` ratio, sometimes called the Hastings correction, is what keeps the chain honest when the proposal is not symmetric. Symmetric means `q(theta' | theta) = q(theta | theta')` — true for `theta' = theta + noise` with zero-mean symmetric noise, which is the common random-walk case. Then the correction is 1 and the rule reduces to the original Metropolis form `min(1, p(theta') / p(theta))`. With an asymmetric proposal — a lognormal step for a positive parameter, an independence proposal from some fitted approximation, a move that reflects off a boundary — the correction is not 1, and dropping it is a genuine bug rather than a rounding issue. The chain will still run and still look plausible, but it will converge to the wrong distribution, because the asymmetry biases which moves get offered and nothing compensates for it. The acceptance ratio is constructed so that the chain satisfies detailed balance with respect to the target; the correction term is exactly the piece that restores balance when proposals are lopsided. ## Working in logs On any real model, `p_unnorm` is a product of many densities and underflows to zero in double precision long before the parameters get extreme. Every implementation therefore computes a log unnormalised posterior — the log-likelihood plus the log-prior, both sums — and tests ``` log(u) < log p_unnorm(theta') - log p_unnorm(theta) ``` which is algebraically equivalent to `u < alpha` for the symmetric case and numerically stable. A parameter value outside the prior's support is handled by returning minus infinity from the log target, which makes the difference minus infinity and the proposal certain to be rejected. ## Reading the acceptance behaviour Two cases are worth naming explicitly. If the proposal has higher unnormalised posterior density than the current point, the ratio exceeds 1, `alpha` is 1, and the move is always accepted — the chain always goes uphill when offered. If the proposal is worse, it is accepted with probability equal to the ratio, so a slightly worse point is usually accepted and a much worse point is usually rejected. That downhill acceptance is not a flaw to be tuned away: it is what makes the chain explore a distribution rather than climb to the mode. A candidate who says the rule is accept-if-better is describing hill climbing, which finds a maximum and tells you nothing about posterior spread. ## What the trick does not buy you The cancellation makes the sampler runnable, not efficient. Nothing in it guarantees that the chain moves usefully — that depends on how proposals are generated, and a badly scaled proposal produces a valid chain that explores hopelessly slowly. The cancellation also gives you no access to `Z` itself; if you actually need the marginal likelihood, for example to compare models, plain Metropolis-Hastings output does not hand it to you, and you need a different technique.