What does per-class loss weighting change when one intent has 200,000 training examples and others have 20?

## The problem weighting is trying to solve A classifier is trained by minimising an average over examples, most often cross-entropy: for one example with true class `c`, the loss is `-log p_c`, where `p_c` is the probability the model assigns to the true class. Training minimises the mean of that quantity over the training set. "Mean over the training set" is the whole difficulty. If a 300-intent support router has one head intent with 200,000 utterances and a long tail of intents with 20 utterances each, then roughly all of the terms in that mean come from the head. A model that predicts the head intent confidently and treats every tail intent as noise achieves a very low mean loss. Gradient descent has no reason to look for anything better: the tail contributes a negligible share of the gradient at every step. This is not a bug in the optimiser, it is the objective you wrote down. ## What the weighting actually does Per-class weighting changes the objective to a weighted mean: each example's loss is multiplied by a weight `w_c` that depends only on its class, and the weighted terms are averaged. A common choice is `w_c` proportional to `1 / n_c`, where `n_c` is the number of training examples in class `c`; that makes every class contribute the same total loss mass regardless of how many rows it has. Per-sample weighting is the same idea with the weight attached to the row instead of the class: upweight a cohort you care about disproportionately, downweight rows from an annotator or a scraping heuristic you distrust. Mechanically, for the usual softmax-with-cross-entropy head, the gradient of the loss with respect to the logits of one example is `p - y` (predicted distribution minus the one-hot target). Weighting scales that whole vector by `w_c`. So a tail example does not push in a different direction than it would have; it pushes in the same direction, harder. ## The consequence people forget: the prior moves Minimising a `w_c`-weighted loss is, in expectation, the same as minimising the unweighted loss on a different data distribution, one where each class's probability has been multiplied by `w_c` and renormalised. That has a precise consequence: the trained model's outputs approximate posterior probabilities under that rebalanced prior, not under the prior your production traffic actually has. If you weight a 1-in-10,000 class up to parity, its predicted probabilities come out orders of magnitude larger than its real-world rate. Whether that matters depends entirely on what the score is for. If the score is only ranked or compared against a cut-off you will tune on validation data anyway, the shift is harmless and often helpful, because the model now spends capacity on the classes you care about. If a downstream system consumes the number as a probability - expected-cost arithmetic, a routing budget, a displayed confidence - the number is now wrong in a systematic direction, and you must either recalibrate on data drawn at the natural rate or subtract `log w_c` from the corresponding logit at inference to undo the shift. ## What it cannot fix Weighting redistributes influence over the data you have; it does not add data. Twenty utterances for an intent contain twenty phrasings. Multiplying their loss by 10,000 makes the network fit those twenty phrasings extremely well, which is another way of saying it overfits them: training macro-recall climbs while validation macro-recall for the same intents barely moves, or moves and then collapses. The same amplification applies to their defects. If two of the twenty rows are mislabelled, weighting has just made those two mistakes thousands of times more influential than any head-class row. There is also a practical scaling detail. If the weights do not average to about 1, the magnitude of the whole loss changes, and with plain stochastic gradient descent or momentum that behaves like a change in learning rate on top of the rebalancing you intended. Normalise the weights so their mean over the training distribution is 1, and you can change the weighting scheme without simultaneously re-tuning the step size. Adaptive methods such as Adam divide by a running estimate of the gradient's second moment and are therefore largely insensitive to a *global* rescaling of the loss, but they do not undo the *relative* imbalance between examples, which is the part you actually wanted. ## How to tell whether it worked Judge it on a metric that gives rare classes a vote - macro-averaged recall or per-class recall for the tail, reported on a held-out set drawn at the natural rate. Overall accuracy will usually get slightly worse when weighting works, because you have deliberately traded head-class errors for tail-class coverage. A candidate who reports that accuracy dropped and concludes the weighting failed has misread the whole exercise.