For an autoencoder's reconstruction loss, when do you use cross-entropy over squared error?

## Both losses are log-likelihoods A reconstruction loss is not an arbitrary distance. Writing the decoder as a conditional distribution over outputs given the code, and minimising the negative log-likelihood of the true input under it, produces the familiar losses directly. Assume each output dimension is Gaussian with mean equal to the decoder output and a fixed shared variance. The negative log-likelihood of one dimension is `(x - x_hat)^2 / (2*sigma^2)` plus a constant that does not depend on the parameters. Minimising it is minimising squared error. The fixed variance is why plain squared error weights every dimension equally, and it is the assumption most people make without noticing. Assume instead each output dimension is Bernoulli with probability equal to the decoder output. The negative log-likelihood of one dimension is `-(x*log(x_hat) + (1-x)*log(1-x_hat))` — binary cross-entropy. This requires `x_hat` strictly inside (0,1), which is why it is paired with a sigmoid output, and requires the target `x` to lie in [0,1] for the expression to be meaningful. Both also assume the output dimensions are **conditionally independent given the code**: the total loss is a sum over dimensions with no term coupling neighbouring pixels or adjacent frames. All structure has to arrive through the code; the loss itself never rewards getting the relationship between two outputs right, only each one separately. ## When each one applies **Bernoulli cross-entropy** fits data that is genuinely bounded to [0,1] and is naturally read as an intensity or a probability. Handwritten-digit pixels rescaled from 0-255 to [0,1] are the canonical case. Note that the targets need not be 0 or 1 — the expression is perfectly well behaved at `x = 0.37`, is still minimised at `x_hat = x`, and is a valid training signal. **Squared error** fits unbounded real values: log-mel spectrogram frames, standardised sensor readings, embeddings, residuals. Applying cross-entropy here is not a stylistic mistake but an undefined one — `log(x_hat)` weighted by a target of `-3.1` has no meaning, and clipping the data into [0,1] first destroys exactly the magnitude information you were trying to reconstruct. ## The gradient argument There is a second, practical reason to prefer the matched pairing. Let `z` be the decoder's pre-activation and `p = sigmoid(z)`. With cross-entropy, `dL/dz = p - x`. The sigmoid derivative cancels analytically, so an output that is confidently wrong — `p` near 1 when `x` is 0 — produces a gradient near 1, the largest correction available. With squared error through the same sigmoid, `dL/dz = (p - x) * p * (1 - p)`. The second factor approaches zero exactly when the output is saturated, so the most wrong outputs generate the smallest gradients and training crawls out of saturation slowly. This is the same saturation argument that makes cross-entropy the default for classification, applied to reconstruction. ## Reading the numbers The two losses are not on a common scale and their floors differ. Squared error reaches zero at a perfect reconstruction. Cross-entropy with a fractional target does not: at `x_hat = x = 0.37` the per-dimension loss equals the binary entropy of 0.37, roughly 0.66 nats, and the total loss floor is the sum of those entropies over all dimensions. A model reporting a cross-entropy of 0.7 per pixel may be at the optimum or far from it; you cannot tell from the absolute number. Compare within a loss, across models trained on the same data, never across losses. ## Scaling, weighting and the choice you make implicitly Because squared error assumes a single shared output variance, every dimension contributes in its own raw units. If one sensor channel swings over hundreds and another over hundredths, the loud channel dominates the total error and the code will spend nearly all its capacity there. Standardising each channel before training, or attaching per-dimension weights, is the fix — and either amounts to declaring a per-dimension output variance, which is what the fixed-variance Gaussian assumption was hiding. The same logic covers mixed inputs. A record with continuous and binary fields wants a per-field loss — squared error for the continuous ones, cross-entropy for the binary ones — summed with weights you choose deliberately, because the relative weighting decides which fields the bottleneck prioritises. ## The short version for an interview Say what distribution you are assuming, check that the target range is compatible with it, and mention the gradient behaviour of sigmoid plus cross-entropy. Then add the caveat about comparing loss values across losses. That sequence answers the question and shows you know the losses are modelling choices rather than defaults.