In diffusion modelling, when is a fixed Gaussian forward process the wrong fit for your data?

## What the fixed forward process quietly assumes The corruption is defined once, before any training, as repeated scaling by `sqrt(1 - beta_t)` plus Gaussian noise of variance `beta_t`. Three assumptions ride along with that definition. **Continuity and a meaningful metric.** Adding a small Gaussian perturbation must produce something that is a slightly worse version of the original, and directions in the space must be comparable. This is true of pixels, of spectrograms, of joint angles and of end-effector poses. It is not true of token identities: perturbing a one-hot vector or a discrete embedding produces a point between symbols with no interpretation, and the reverse chain then spends its capacity climbing back to the nearest symbol rather than modelling structure. Discrete corruption processes exist for that setting, but they are a different design decision, not a normalisation trick. **Comparable per-dimension scale.** The corruption magnitudes are shared across all dimensions. If one feature is ten times larger than the rest, the small ones are already indistinguishable from noise while the large one still carries clear signal, so the effective noise level differs per dimension and much of the chain is wasted. Standardising features is the fix, and the variance-preserving property also assumes roughly unit-scale data - it is what keeps `x_t` on one scale across the chain. Heavy-tailed features resist this: after standardisation the rare extremes still dominate, and Gaussian corruption around a heavy-tailed marginal never reaches the intended prior cleanly. **A budget of many evaluations per sample.** Training touches one step per example, but generation runs the reverse chain. If your latency budget permits exactly one forward pass, this family is disqualified on cost before any statistical argument matters. ## The argument *for*, and where it is strongest The decisive question is whether the conditional distribution you are modelling is genuinely multimodal. A model trained with squared error onto a single output returns the conditional mean. When several distinct answers are valid, that mean is often not one of them. Robot action trajectories make the case cleanly. Suppose a policy must produce a short sequence of end-effector waypoints toward a goal, and the demonstrations show two acceptable routes - around the left of an obstacle and around the right. A regression head trained on both averages them into a path through the obstacle: an output that appears in no demonstration and is physically wrong. Apply the same fixed Gaussian corruption to the action sequence instead - the trajectory is a modest-dimensional continuous vector, so the closed form and the noise-prediction objective transfer verbatim - and sampling the reverse chain returns *one* route rather than the average of two. The conditioning is the observation, injected exactly as a class or text embedding would be, with the same null-token dropout available for guidance. Nothing about the method is image-specific; only the tensor shape changes. That reframing also sharpens when *not* to use it. If demonstrations are unimodal - one route, one grasp, one canonical answer - the mean is a valid answer and a direct regressor gives it in one pass at a fraction of the cost. Reaching for a generative family when the conditional is unimodal is spending an iterative sampling budget on variability nobody wanted. ## Hard constraints deserve their own line The reverse chain has no built-in mechanism for feasibility. Joint limits, collision-free geometry, non-negativity, sum-to-one totals - none of these are enforced by a Gaussian reverse step, and samples can land just outside a feasible set. The usual remedies are projecting or clipping after each step, encoding the data in a parameterisation where the constraint is automatic, or filtering samples downstream. Whichever you choose, it is a design obligation you accept alongside the family, and a good answer names it rather than assuming the model will respect physics because the training data did. ## The decision, compactly Ask four questions in order. Is the representation continuous and roughly comparably scaled, so that isotropic Gaussian noise is a meaningful corruption? Is the conditional distribution genuinely multimodal, so that a mean is not an acceptable answer? Can you afford an iterative sampling budget at serving time? Do hard feasibility constraints exist that the reverse chain will not enforce for you? Four yeses with a plan for the fourth is a strong fit. A no on the first sends you to a corruption process designed for that data type; a no on the second sends you to a plain regressor; a no on the third sends you to a single-pass generator family. ## What separates a principal answer Juniors and mid-level candidates tend to answer 'diffusion works for images'. The senior answer states the assumptions. The principal answer treats the corruption process as a *design choice* with a domain contract - it presumes a metric, a scale and an evaluation budget - and reasons about what to do when the contract is violated, rather than normalising the data until the formula runs and hoping the samples are sensible.