Does label smoothing fix an overconfident classifier's calibration?

## What is being claimed A classifier is calibrated if, among the predictions it makes with confidence 0.8, about 80 percent are correct. Two tools make this concrete. A **reliability diagram** bins held-out predictions by their top-class confidence and plots bin accuracy against bin confidence; a perfectly calibrated model lies on the diagonal, an overconfident one sits below it. **Expected calibration error (ECE)** collapses that picture into one number: the sample-weighted average of the absolute gap between accuracy and confidence across bins. Make it concrete with a 5-grade diabetic-retinopathy classifier. Its average top-1 confidence is 0.99; its held-out accuracy is around 0.85. Nearly every prediction lands in the top confidence bin, that bin's accuracy is 14 points below its confidence, and ECE is roughly that gap. Clinically this is the worst kind of failure: a grader triaging by confidence sees no signal, because the model reports 0.99 on the cases it gets wrong too. ## Why smoothing moves the number One-hot training has no finite optimum — the correct logit keeps rising relative to the others for as long as you train — so the reported confidence saturates against the floating-point ceiling regardless of how hard the example was. Smoothing gives the objective a finite optimum. At `epsilon = 0.1` over `K = 5`, the uniform component puts 0.02 on every class and the target's top entry is 0.92, so the model is being trained toward a maximum confidence near 0.92 rather than 1.0. If real accuracy is around 0.85, the fitted confidence lands much closer to it and ECE drops sharply. This effect is well documented across image classifiers, and it is the main reason smoothing is discussed as a calibration tool at all. ## Why it is not a calibration method Three limits are worth being able to state. **It is global, not per-example.** Calibration requires that confidence *varies with* difficulty: near 1.0 on an obvious case, near 0.5 on a genuinely ambiguous one. Smoothing applies the same ceiling to every example. It compresses the confidence range rather than reshaping it, so a model can have a good aggregate ECE while its ordering of easy against hard cases stays as poor as before. **It overshoots.** The ceiling depends on epsilon and `K`, not on accuracy. Put epsilon 0.2 on a model that is genuinely 97 percent accurate and you have trained it toward a ceiling below its own hit rate: the reliability diagram flips above the diagonal, and the model is now underconfident. That is a real cost, because a downstream cascade that routes low-confidence cases to a human will now route away work the model would have got right. **It changes the model.** Smoothing is a training-time objective change, so it moves the decision function, the features and the accuracy along with the probabilities. A post-hoc rescaling of scores fitted on a held-out split is monotone: it leaves the argmax and hence the accuracy exactly as they were, and it can be refitted per deployment without retraining. Those are different instruments. The honest position is that smoothing improves probability quality as a side effect of a regularizer, and if calibrated probabilities are the actual product requirement you should still measure and, where needed, recalibrate after training. ## How to measure it properly - Use a held-out split the smoothing run never touched; training-set calibration is meaningless here. - Report the reliability diagram alongside ECE, and state the bin count — ECE is sensitive to binning and a single number hides where the failure lives. - Break calibration out per class. On a 5-grade ordinal problem the rare severe grades are exactly where confidence is worst and where an aggregate number, dominated by the common grades, will not show it. - Sweep epsilon and plot accuracy and ECE together. They do not peak at the same value, and choosing on accuracy alone is how a team ends up underconfident by accident. - Re-tune any operating threshold. If a triage rule was set at confidence 0.95 against an unsmoothed model, that rule may be unreachable once the ceiling is 0.92. ## The metric that gets worse One consequence surprises people. Held-out log-likelihood, and its exponential per-token form perplexity, score the probability the model assigned to the observed label. A smoothed model is trained never to give it full mass, so that number gets worse even as the argmax decisions and the calibration improve. That is expected, not a defect — but it means you cannot use held-out likelihood as your model-selection metric while sweeping epsilon.