What is the difference between random oversampling and random undersampling of an imbalanced training set?

## The problem being attacked An imbalanced training set is one where the label you care about is rare: 10,000 churners against 200,000 non-churners in a telecom table, or 400 defective parts in 50,000 inspected units. Most learners minimise an average loss over rows, so a class that supplies 5% of the rows supplies about 5% of the pressure. The model can score well by leaning toward the common label. **Resampling** attacks this by changing the composition of the training rows so the class ratio the learner optimises against is closer to balanced. The two crudest forms are random oversampling and random undersampling. ## Random oversampling: duplicate the minority Draw minority rows uniformly at random **with replacement** and append the copies until the target ratio is reached. Nothing is invented — repeating the same 400 defective parts twenty times still describes exactly 400 distinct defects, and gets you to 8,000 minority rows against 49,600 good ones, which is not even balanced. Mechanically, a duplicated row contributes to the objective once per copy. In a tree, node class counts change, so different splits are chosen and a deep, pure leaf drawn tightly around a duplicated point becomes easy to justify — including around a mislabelled one. In a margin- or distance-based model, the copies occupy the *same* coordinates as the original, so they cannot enlarge the region the minority covers; they only raise the price of getting that one location wrong. Arithmetically, appending a row twenty times is the same as counting it twenty times in the loss. Consequences worth naming: - **Memorisation.** With enough capacity the model fits the individual duplicated rows, so training scores look excellent and held-out scores do not move. - **Cost.** The training set grows. Balancing the telecom table by oversampling produces 400,000 rows instead of 210,000. - **No new coverage.** Minority regions that were never sampled stay unsampled. ## Random undersampling: delete the majority Drop majority rows at random until the ratio is reached. Balancing the telecom table means keeping 10,000 of the 200,000 non-churners and discarding roughly 95% of them. - **It is fast.** The training set falls to 20,000 rows, which matters when you are fitting many candidate models. - **The loss is real and one-directional.** Unusual-but-genuine majority rows — the rare corners of "normal" — may vanish entirely, so the model's picture of the common class gets thinner and the boundary shifts noticeably between random draws. That is added variance, not just lost accuracy. - **The standard mitigation is not to discard permanently.** Train several models on different random majority subsets and combine them, so every majority row is seen by some ensemble member. Ensemble undersampling methods such as EasyEnsemble are built on exactly this idea. ## What neither one does Neither adds information about the rare class. What they change is the ratio the objective is averaged over, which makes the model more willing to emit the rare label. A direct consequence: the scores a resampled model produces reflect the resampled ratio, not the population rate — so a raw probability from such a model should not be read as a real-world frequency. ## Choosing how far to go Full balance is a convention, not a requirement, and on extreme ratios it is a violent intervention: a 0.8% defect rate needs about 124 copies of every defect to reach 1:1. Partial resampling — lifting the minority to 10% or 20% — is usually the better starting point, and the target ratio deserves to be tuned like any other hyperparameter and judged on evaluation data you never resampled. ## How to answer in an interview State the mechanics in one line each (copy versus delete), then name the cost of each (memorisation and size versus lost majority variation and instability), then say what you would actually do: check whether the imbalance is even hurting before intervening, prefer undersampling when the majority is large and redundant, prefer oversampling when every row is precious, and treat the resampled ratio as a tuned knob rather than a fixed 1:1 rule. SMOTE, which interpolates new minority points instead of copying them, is the natural next step from random oversampling.