Should a classifier that reads an RNN's final hidden state use pre- or post-padding?
answer
- last index is not the last real step
- pads at the end keep updating the state
- pads at the front shift every position
- read the state at length minus one
- or carry the state through masked steps
basics
~20 sPost-padding puts pad steps last, so the state at the final index is the state after the pads, not after the last real token. Pre-padding hides that, but the real fix is to read each sequence's state at index length minus one.
solid answer
~50 sWith post-padding the pads sit after the real steps, so for every sequence shorter than the batch maximum the state at the last index is the state after a run of pad steps, which has drifted away from what the last real token produced. Reading it hands the classifier a length-dependent corruption that hurts short sequences most. Pre-padding puts the pads first, so the last index is always a real token and the naive readout is correct; that is why it is the usual recipe when the readout can only take the last position. The cleaner fix keeps post-padding and gathers each row's state at index `L_i - 1`, or skips pad steps in the recurrence so the state carries through unchanged. With masked pooling or per-step tagging, the side affects target alignment, not correctness.
go deeper
Know that padding can go before or after the real steps, and that with pads at the end the last row of the array is usually a pad step rather than the end of the sentence.
Trace what the recurrent state does during a run of trailing pad steps, and name the fix: read each sequence's state at index length minus one instead of at the last index.
Show that you would rather fix the readout than the padding side, and describe the symptom you would look for - short sequences scoring worse, and metrics that move when the batch composition changes.
Treat batch layout as an interface contract. Padding side leaks into target alignment, evaluation and serving code, so decide it once and write it down rather than leaving it per model.
## The setup A sequence classifier of the sequence-to-label kind runs a recurrent layer over the input and feeds one vector to a small head. The usual choice for that vector is "the final hidden state" — the state after the model has seen the whole input. In a padded batch, "the final hidden state" and "the state at the last index of the array" are not the same thing, and the gap between them is exactly what this question is about. ## What post-padding does Post-padding is the common default: real steps first, pads appended to reach the batch's longest length. Now consider a batch padded to 900 steps containing a 30-step sequence. Index 899 for that row is the state after 870 pad steps. Because a pad step is still a recurrent update — `h_t = tanh(W h_(t-1) + U x_pad + b)` runs whatever `x_pad` is, including zero — the state has been pushed 870 times toward a fixed point of that map. Whatever the last real token contributed has largely been washed out. The symptom is diagnostic: accuracy that degrades with how short the sequence is relative to the batch it landed in, and results that change when you reshuffle, because reshuffling changes which long sequence sets each batch's padded width. It is not a crash and not a loud loss spike; it is a quiet accuracy ceiling. ## What pre-padding does Pre-padding puts the pads at the front, so every row ends on a real token and the state at the last index really is the state after the last real step. This is why pre-padding is the standard advice for last-state classifiers: it makes the naive readout correct with no per-row bookkeeping. It is not free. The model now starts each short sequence by consuming a long run of pads, so by the time the first real token arrives the state is not the initial state but wherever the pad run pushed it. A gated recurrent cell mostly recovers, since its gates can overwrite the state once real input arrives; a plain tanh cell is more exposed. And pre-padding shifts alignment: the real step at position `t` of a sequence sits at array index `t + (T - L_i)`, a different offset per row. Anything that reads positions — per-step targets, a step-indexed feature, a causal convolution's alignment — must be offset the same way, and getting that wrong is a much harder bug to see than the one you were trying to avoid. ## The fix that makes the question go away Two options, both better than choosing a side by folklore. **Gather by length.** Keep post-padding, carry each sequence's true length `L_i`, and read the state at index `L_i - 1` for each row. This is correct regardless of how much padding the batch carries, needs no alignment offsets, and keeps the array layout that per-step targets already assume. **Skip the pad steps.** Have the recurrence copy the previous state through when the mask is zero instead of updating it: `h_t = m_t * update + (1 - m_t) * h_(t-1)`. Then the state at the last index equals the state after the last real step by construction, the naive readout becomes correct, and you also stop spending compute on pad steps. With either in place, padding side becomes a layout preference rather than a correctness question. ## When the side genuinely does not matter If the head reads a masked pooled vector over time rather than one position, or if the task is per-step tagging with a masked loss, correctness does not depend on which end the pads sit at — the mask removes them either way. What still depends on the side is *alignment*: per-step targets must be padded on the same side as the inputs, or step `t`'s prediction gets scored against step `t`'s label from a different offset. That mismatch is silent, produces plausible-looking loss values, and is one of the classic reasons a tagger trains to mediocre accuracy for no visible reason. ## The judgment being tested The weak answer picks a side and stops. The strong answer says: the padding side is a workaround for a readout that cannot index per row, the real question is what "final state" means for each sequence, and once the readout is length-aware the choice reduces to whichever layout keeps target alignment simplest — usually post-padding, because it matches how lengths and per-step targets are naturally stored.
- How do you get the correct final state without changing which side you pad on?Carry the true length of each sequence alongside the batch and read that row's state at index `L_i - 1`. It is a per-row gather, correct for any padded width, and it leaves the array layout alone so per-step targets keep their alignment. The alternative is to make the recurrence copy the state through at masked steps, which makes the last index correct by construction.
- For per-step tagging with a masked loss, does the padding side still matter?Not for correctness of the loss — the mask drops pad positions either way. It matters for alignment: inputs, targets and mask must all be padded on the same side, or step t's prediction is scored against the wrong step's label. That mismatch produces a plausible loss curve and mediocre accuracy, which makes it much harder to find than an obvious crash.
- What symptom would make you suspect a post-padded batch is corrupting the final state?Accuracy that falls off with sequence length relative to the batch — short inputs doing worse than long ones — and metrics that move when you reshuffle or change the batch size, since both change each batch's padded width. A quick check is to evaluate one sequence alone, unpadded, and compare with its prediction inside a wide batch.
saying these in an interview costs you the question
- Says the padding side never matters as long as you pad
- Reads the last array index and calls it the final state
- Assumes trailing zero steps leave the state untouched
- Pre-pads the inputs but post-pads the per-step targets
- Fixes the side instead of making the readout length-aware