Why does a caption decoder score well teacher-forced but degenerate when it feeds itself?

## The two conditional queries An autoregressive decoder is a single function, but training and deployment probe it at different inputs. - **Training (teacher forced):** the model is asked, at every position, what follows a *ground-truth* prefix. The prefixes it sees are drawn from the data distribution of real target sequences. - **Generation (free running):** the model is asked what follows *its own* prefix. Those prefixes are drawn from the model's own output distribution. Maximum likelihood under teacher forcing fits the first query. Nothing in the objective constrains the second. This mismatch is **exposure bias**: the model was never exposed to the inputs it will actually face. ## Compounding error, stated precisely Suppose the decoder emits a wrong token at step 3. Every later step now conditions on a prefix containing that token. If such a prefix has essentially zero probability under the training distribution of target sequences, the model has no data-driven constraint there at all — its output is whatever the learned function extrapolates to. The conditional estimate at step 4 is therefore less reliable than at step 3, which makes another wrong token more likely, which pushes the prefix further from anything seen in training. The per-step error probability is not constant along the sequence; it grows, because the conditioning context itself is drifting. That is the compounding part. It is why the degradation is worst at the tail of long generations and why short outputs often look fine. ## The symptoms **Degenerate repetition.** An image-captioning decoder that scores well with ground-truth prefixes can collapse to "a man a man a man" as soon as it feeds itself. Once a repeated bigram is in the prefix, the strongest local pattern in that off-distribution context is the repetition itself, and the loop is self-sustaining. The model is not confused about the image; it is stuck in a region where its own history dominates the conditioning. **Unrecoverable structural violations.** A SMILES molecule generator trained on valid strings emits an unbalanced opening parenthesis at step 12. Every valid continuation in the training data closed the brackets it opened, so the model has never learned what to do from a state where the count cannot be repaired, and the rest of the string is garbage. The constraint is global, but the model only ever learned it from prefixes that already satisfied it. **Drift and hallucinated detail.** In longer generation the output stays fluent but slowly stops corresponding to the source, because fluency is locally reinforced by the model's own prefix while grounding in the source input is not. ## Why your metrics missed it Teacher-forced validation loss and perplexity are computed exactly like the training loss: at every position the model is handed the correct prefix. They are honest measurements of the teacher-forced conditional and they are useful — they will catch under-fitting, a broken data pipeline, or over-fitting to the training set. What they cannot do is measure behaviour on self-generated prefixes, since by construction the model never generates anything during that evaluation. A model can therefore improve its validation perplexity for several epochs while its free-running output gets worse. The operational fix is to make free-running evaluation a first-class metric, not an afterthought: - Run the decoder autoregressively on held-out inputs and score the *whole sequence* against the reference with a sequence-level metric appropriate to the task, plus cheap structural checks such as validity rate, output-length distribution, and repeated-n-gram rate. - Track it on the same cadence as the loss so you can see the two curves diverge instead of discovering it after a release. - Report the gap between teacher-forced perplexity and the free-running score as its own quantity. A widening gap is the signature of exposure bias; a model with a small gap is one whose likelihood numbers you can trust as a proxy. ## Diagnosing it in a real run When someone reports "great validation loss, terrible outputs", separate three candidate causes before blaming exposure bias: 1. **An evaluation harness bug** — the generation path silently still receives ground-truth tokens, or feeds the wrong shift, or starts from the wrong token. Verify that the free-running path is genuinely self-fed. 2. **A training/serving discrepancy** — different tokenisation, different special tokens, or different preprocessing between the two paths. 3. **Genuine exposure bias** — which you confirm by seeding generation with a correct prefix of increasing length and watching how far the model gets before degenerating. If a longer forced prefix reliably buys more good tokens, and quality falls off after the forcing stops, the diagnosis is the distribution shift rather than a plain capability failure. ## Perspective Exposure bias is a real and well-defined mismatch, but its practical severity varies. It is worst where outputs are long, where the target has global structure that a single early token can violate, and where the model is small or the data thin. It shrinks — though it does not vanish — as models get more capable, because a stronger model makes fewer early mistakes and generalises better to prefixes it has not seen. That is why the first response to a degeneration report should be measurement of the gap, not an immediate change of training objective.