MCMC Samplers

## The problem MCMC exists to solve Bayesian inference states the answer in one line: ``` p(theta | y) = p(y | theta) * p(theta) / Z, Z = integral over theta of p(y | theta) p(theta) d theta ``` Here `theta` is the vector of unknown parameters, `y` is the observed data, `p(y | theta)` is the likelihood, `p(theta)` is the prior, and `Z` (the marginal likelihood, or evidence) is the constant that makes the posterior integrate to one. The numerator is cheap: for any candidate `theta` you can evaluate likelihood times prior directly, usually as a sum of log terms. The denominator is the hard part, because it is an integral over the entire parameter space. In a handful of textbook cases the integral has a closed form — a conjugate prior paired with its likelihood returns a posterior in the same family, and you can read the answer off. Realistic models rarely cooperate. Take a logistic regression with 12 coefficients and a normal prior on each: the likelihood involves a product of logistic terms, the prior is Gaussian, and the product integrates to nothing you can write down. Numerical quadrature does not rescue you either, because grid cost grows exponentially in dimension: a coarse 20-point grid per coordinate already means 20^12, roughly 4 * 10^15 evaluations, for a model most people would call small. ## The reframing: you rarely want the density itself The key observation is that a density function is almost never what a question actually needs. What you report is a summary: - a posterior mean or median for a coefficient, - a probability such as `P(theta_3 > 0.5)`, - a credible interval, - the posterior mean of some derived quantity, such as a predicted probability. Every one of those is an expectation or a quantile under the posterior. And expectations can be estimated from draws without ever knowing the density's scale. If `theta^(1), ..., theta^(S)` are draws whose long-run distribution is the posterior, then for any function `g`: ``` E[g(theta) | y] is approximated by (1/S) * sum over s of g(theta^(s)) ``` Set `g(theta) = theta_j` and you get the posterior mean of coefficient `j`. Set `g(theta) = 1 if theta_j > 0.5 else 0` and the average is exactly the fraction of draws above 0.5 — an estimate of `P(theta_j > 0.5 | y)`. Sort the draws of one coordinate and take the 5th and 95th percentiles and you have a 90 percent credible interval. Nothing in any of these calculations refers to `Z`. ## Where the draws come from That leaves the question of how to draw from a distribution you can only evaluate up to an unknown constant. Markov chain Monte Carlo answers it by giving up on independent draws. Instead of sampling from the posterior directly, you build a rule for stepping from the current parameter vector to the next one, designed so that the chain spends time in each region of parameter space in proportion to that region's posterior probability. Run it long enough and the sequence of visited states behaves, for averaging purposes, like a sample from the posterior. Metropolis-Hastings, Gibbs sampling and Hamiltonian Monte Carlo are three such stepping rules; they differ in how a move is proposed and whether it can be rejected, not in what they target. Crucially, every one of these rules only ever needs the unnormalised posterior — likelihood times prior — because the moves depend on comparing two candidate values, and the unknown constant scales both sides identically. ## What you give up MCMC output is an approximation with two distinct error sources, and a candidate who says otherwise is glossing over the hard part. First, Monte Carlo error: with a finite number of draws, the sample average is not the true posterior mean. More draws shrink this error, and you should report summaries with enough draws that the estimate is stable to the precision you plan to quote. Second, dependence: successive states of the chain are correlated by construction, since each is a step away from the last. Ten thousand MCMC draws therefore carry strictly less information than ten thousand independent draws, sometimes dramatically less. That is why raw draw counts are a poor measure of how much you have actually learned. Third, and easy to forget: the guarantee is asymptotic. A chain started far from the bulk of the posterior needs time to reach it, and a chain that never finds a second mode will produce confident, wrong summaries. Deciding whether a particular run has actually got there is a separate discipline from choosing a sampler. ## The practical takeaway MCMC converts an intractable integration problem into a simulation problem. You give up closed-form answers and independent samples; you get the ability to work with any posterior you can evaluate up to a constant, at whatever dimension the model requires. For most applied Bayesian work, that trade is the whole reason the method exists.