Why shuffle the training rows before each epoch of stochastic or mini-batch gradient descent?

## What shuffling is protecting you from Stochastic and mini-batch gradient descent update the weights from whichever rows they happen to see next. That makes the *order* of the training file part of the algorithm. If consecutive rows resemble each other — because the file is sorted, grouped or exported in arrival order — then consecutive updates all push the weights in the same direction, toward whatever fits that group, and the fit becomes a tour of local sub-populations instead of a march toward the overall optimum. Reshuffling the row order before every epoch breaks that structure: each batch becomes an approximately random sample of the training set, so the batch-averaged gradient is a fair estimate of the full-data gradient rather than the gradient of one segment. ## The concrete failure Consider a training file sorted by customer id. Rows for a single customer sit adjacent, so with a batch size of a few hundred, whole batches can be dominated by one customer or one contiguous band of ids — which in practice correlates with signup cohort, region or product line. The visible symptom is a **sawtooth loss curve**: within each epoch the loss falls while the fit chases the current segment, then jumps when the pass reaches a segment with different characteristics, and the same pattern repeats identically every epoch because the order never changes. The curve looks like a periodic waveform locked to the epoch boundary. The pathological version is a file sorted by the target. Fit logistic regression on a file where every negative row comes before every positive row and the early updates drive the intercept toward predicting "always negative", the later ones drag it back, and the model that survives the last epoch is whichever extreme the pass ended on. Order also gives the final batches of the final epoch outsized influence on the weights you ship, simply because nothing moved the weights afterwards. ## Why *every* epoch, not once Shuffling once before training already fixes the worst problem — batches are no longer homogeneous. But a single shuffle freezes one partition of the data: the same rows share a batch in every epoch, so the same correlated gradient estimate is presented over and over. Reshuffling each epoch varies the composition of the batches, so any accidental structure in one partition does not repeat, and the sequence of gradient estimates the fit sees is far closer to independent draws. Reshuffling is cheap relative to a pass over the data, so there is no reason not to. ## Full-batch descent is exempt Full-batch gradient descent averages over every row before it moves the weights, and addition does not care about order (up to floating-point rounding, which is negligible here). Shuffling a full-batch fit changes nothing. This is a good discriminating question: a candidate who says "always shuffle, in every algorithm" has memorised a rule; a candidate who says "shuffling matters exactly when updates are computed from subsets" understands why the rule exists. ## Related ordering hygiene Two things sometimes confused with per-epoch shuffling but distinct from it: - **Time-ordered data.** If rows carry a time dimension and the evaluation is "train on the past, score the future", the *split* must respect time. Shuffling for batching happens *within* the training partition, after that split — it never means shuffling rows across the split. - **Deterministic runs.** Shuffling is what makes two runs of the same fit produce slightly different weights. Fixing the random seed makes the shuffle reproducible without removing it, which is the right way to get run-to-run comparability. ## What to say in an interview Shuffling is not a ritual. It is the step that makes each mini-batch gradient a fair sample of the full gradient. Without it, correlated batches produce a sawtooth loss curve, the ordering biases the final weights toward whatever the pass ended on, and in the sorted-by-label case the fit can be badly wrong. Reshuffle every epoch, shuffle only within the training partition, and note that full-batch descent is unaffected.