How does scheduled sampling mitigate exposure bias, and what does it cost?

## The idea Teacher forcing trains only on ground-truth prefixes; free running from scratch gives almost no learning signal early on. Scheduled sampling sits between them. At each decoder step during training you draw a coin. With probability `eps` you feed the ground-truth previous target token, exactly as in teacher forcing. With probability `1 - eps` you feed the token the model itself produced at the previous step, obtained by an argmax over the previous step's distribution or by sampling from it. The loss is unchanged — cross-entropy against the true token at each position — only the *inputs* change. The schedule is the second half of the method. Training starts at `eps = 1` (pure teacher forcing) and the probability of consuming the model's own token is annealed from 0 upward as training progresses, commonly toward roughly 0.5 rather than to full self-feeding. The schedule can be linear, exponential, or inverse-sigmoid in the step count. The intent is a curriculum: give the model a clean signal while it is still incompetent, then progressively expose it to its own outputs so it learns to continue from a prefix containing its own mistakes rather than only from perfect history. ## Why it helps The defect it targets is that a teacher-forced model has never been asked "what comes next after a prefix I got partly wrong?". Scheduled sampling makes that question part of training. Empirically it reduces the collapse into degenerate output, and models trained this way tend to recover from an early mistake instead of drifting further off. It also narrows the gap between teacher-forced likelihood and free-running quality, which is the number you actually care about. ## What it costs **A non-differentiable step in the path.** Picking the model's own token requires argmax or a categorical draw. Neither is differentiable, so the usual implementation treats the fed token as a constant and no gradient flows back through it into the step that produced it. The model is thus taught to react to its own mistakes but never gets a gradient telling it not to have made them at that earlier step. **Loss of the known-input advantage.** Teacher forcing's practical appeal is that the decoder's entire input sequence is fixed before the forward pass. Scheduled sampling breaks that: input `t` depends on the model's output at `t-1`, so training re-acquires a step-by-step dependency and the cheap single-pass scoring of a whole target sequence is gone. On a non-recurrent decoder this is a large wall-clock penalty, and it is one reason the technique fell out of favour as decoders became parallel-trainable. **Slower, noisier convergence.** Feeding wrong tokens injects noise into the conditioning context, which is exactly the point, but it also degrades the learning signal. Anneal too aggressively and training stalls; anneal too slowly and you have effectively trained with teacher forcing. The schedule becomes another hyperparameter to tune, and its right setting is task- and dataset-dependent. **A theoretical objection.** The scheduled-sampling objective is not a consistent estimator of the data distribution: the model that minimises it is not guaranteed to be the true conditional model, and cases can be constructed where a model that largely ignores the conditioning prefix scores well. So the method optimises something adjacent to what you want. This does not make it useless in practice, but it means improvements need to be demonstrated empirically on free-running generation rather than assumed. ## Where it sits among the alternatives Other responses to the same mismatch: - **Professor forcing** trains a discriminator to tell teacher-forced hidden-state trajectories from free-running ones and pushes the model to make them indistinguishable, attacking the distribution shift in state space rather than in token space. - **Sequence-level objectives** score a whole generated sequence against the reference and optimise that score directly, so the training signal is computed under the self-fed regime the model will actually run in. These are harder to optimise and higher-variance. - **Prefix corruption as data augmentation** — deliberately perturbing target prefixes so the model sees imperfect history — is a cheaper, cruder version of the same curriculum. - **Doing nothing to the objective.** In practice most modern systems keep plain teacher forcing and absorb the mismatch elsewhere: more and better data, larger models that make fewer early mistakes, and constraints or checks applied at generation time. ## How to answer the interview version A strong answer names scheduled sampling, describes the coin flip and the anneal precisely, and then does the honest part: says what it gives up. The failure mode in interviews is presenting it as the standard fix. It is a well-known historical method that is worth understanding because it makes the mechanism of exposure bias concrete, but a candidate who cannot say why it is rarely the first thing reached for today — the wall-clock cost, the tuning burden, the inconsistent objective — is only reciting the paper.