In a diffusion model, why can't the reverse go from pure noise to data in one step?

## The asymmetry It is tempting to think that because `q(x_t | x_0)` is a single Gaussian jump, the inverse should be a single jump too. The catch is what each direction conditions on. Forward, `x_0` is given, so all the uncertainty is the noise you are about to add and it is Gaussian by definition. Backward, `x_0` is precisely the unknown. The distribution you would need is `q(x_(t-1) | x_t)` with `x_0` marginalised out, and that depends on the entire data distribution. Concretely, at a high noise level a single noisy vector is consistent with a huge set of clean samples. The posterior over them is multimodal - several distinct, mutually exclusive answers, not one bump. No Gaussian describes it, and no single deterministic map can sample from it. ## What a single pass would actually give you A network trained with squared error converges to the conditional mean of its target. So the clean-sample estimate implied by the noise prediction, `x0_hat = (x_t - sqrt(1 - abar_t) * eps_hat) / sqrt(abar_t)`, is `E[x_0 | x_t]` - the *average* over every clean sample compatible with `x_t`. Early in sampling, when `x_t` is nearly pure noise, that average is a smooth, low-contrast blur containing no committed content. This is worth stating out loud in an interview because it is directly observable: inspect `x0_hat` at the first few reverse steps and you see the blur; inspect it near the end and you see a sharp sample. The chain does not fail to produce a one-step answer - it produces the *wrong kind* of answer, the mean instead of a sample. ## Why small steps rescue it The classical result behind diffusion models is that the time-reversal of a diffusion process has the same functional form as the forward process in the limit of small steps. Practically: when `beta_t` is small, `q(x_(t-1) | x_t)` is well approximated by a Gaussian with a mean close to `x_t`. That is what makes the reverse learnable with a simple parameterisation: ``` p_theta(x_(t-1) | x_t) = N( mu_theta(x_t, t), sigma_t^2 * I ) mu_theta(x_t, t) = ( x_t - (beta_t / sqrt(1 - abar_t)) * eps_theta(x_t, t) ) / sqrt(alpha_t) ``` The mean is not an image the network paints from scratch; it is `x_t` with a scaled slice of the predicted noise removed and the result rescaled. Each reverse step is a small, nearly linear correction, which is a far easier function to fit than a map from an isotropic Gaussian onto the data manifold. ## How ambiguity actually gets resolved Think of the chain as narrowing a set. At step `T` the compatible set is essentially the whole dataset. Each reverse step both removes a slice of noise and keeps the process stochastic, so the trajectory drifts toward one region rather than the global average. After a while the compatible set has shrunk to one broad mode - a rough layout, a rough colour scheme - and later steps refine detail within it. The multimodality is spent gradually rather than all at once, which is precisely what a single conditional-mean prediction cannot do. Read through the score-matching lens, the same story: the network estimates the gradient of the log density of the noise-smoothed data. Heavy smoothing gives a simple, almost unimodal landscape that is easy to climb from anywhere; light smoothing gives the sharp, spiky true landscape. Following the gradient from heavy to light smoothing is a continuation method - you solve an easy problem first and track the solution as it hardens. ## The price and the boundary of the argument This is why generation costs many network evaluations while training costs one per example. It is the defining trade of the family: adversarial and invertible-flow models generate in a single pass and pay for it in training stability or architectural constraints; diffusion pays at sampling time and gets a stable regression objective in exchange. One nuance worth stating carefully: 'small steps' is a statement about the validity of the Gaussian reverse approximation, not a claim that any particular number of evaluations is required. The point is structural - the reverse conditional is only Gaussian in the small-step regime, so the chain exists for a mathematical reason and not merely as an implementation habit. ## What a strong answer sounds like Name the conditioning asymmetry, say the word multimodal, note that squared error yields the conditional mean and that the mean of many valid samples is a blur, and finish with the small-step Gaussian approximation that makes each reverse move tractable. That covers the theory and the observable symptom in one breath.