How do you read a multiclass log loss of 1.2 from a 20-language classifier?

## The definition For a classifier that outputs a probability distribution over K classes, multiclass log loss (multiclass cross-entropy) over N rows is ``` log_loss = -(1/N) * sum_i log( p_i[ y_i ] ) ``` where `y_i` is the true class of row i and `p_i[y_i]` is the probability the model assigned to that class. Written the other way, with a one-hot target `t_i`, it is `-(1/N) * sum_i sum_k t_ik * log p_ik`, and since `t_ik` is zero for every class except the true one, all those terms vanish. **Only the true-class probability enters each row's term.** That surprises people: if the true language is Portuguese and the model gave Portuguese 0.30, it does not matter for the loss whether the remaining 0.70 sat entirely on Spanish or was spread thinly over the other 18. The wrong classes influence the loss only indirectly, because the softmax normaliser makes the K probabilities sum to one, so mass given to Spanish is mass taken from Portuguese. ## Reading a number A log-loss value is meaningless without its baselines. - **Perfect and confident**: probability 1 on every true class gives `-log(1) = 0`. - **Uninformed**: a flat `1/20` on every row gives `-log(1/20) = log(20)` ≈ **3.00**. In general `log(K)` — note that this baseline moves with the number of classes, so a log loss of 1.2 is mediocre for 3 classes and good for 20. - **Prior-only**: always predicting the training class frequencies gives the entropy of the label distribution, which is below `log(K)` whenever the classes are unbalanced. This is the honest baseline to beat. So 1.2 on 20 languages sits well below the 3.00 uniform baseline. A useful second reading: because the loss is the mean of `-log p`, `exp(-1.2)` ≈ **0.30** is the **geometric mean** of the probabilities assigned to the true languages. Equivalently, `exp(1.2)` ≈ 3.3 is sometimes called the perplexity — loosely, the model behaves as if it were choosing uniformly among about 3.3 languages rather than 20. ## Why the loss is so sensitive to confident mistakes `-log p` is unbounded as `p` approaches zero. Some values worth memorising: ``` p = 0.5 -> 0.69 p = 0.3 -> 1.20 p = 0.1 -> 2.30 p = 0.01 -> 4.61 p = 0.001 -> 6.91 ``` One row where the model gave the true language 0.001 contributes 6.91 to the sum. In a 1000-row evaluation that single row adds 0.0069 to the average — small — but a hundred such rows add 0.69, which would swamp the difference between two otherwise similar models. This is why a log-loss regression between model versions is often traced to a small slice of confidently-wrong inputs rather than to a broad degradation. Because a probability of exactly zero would make the loss infinite, evaluation clips probabilities into a range like `[1e-15, 1 - 1e-15]` before taking the log. Worth mentioning: it shows you know the metric has a failure mode. ## What log loss tells you that accuracy does not Log loss is a **proper scoring rule**: it is minimised, in expectation, exactly when the reported probabilities equal the true conditional probabilities. Reporting anything other than your honest belief makes it worse. Accuracy only asks whether the argmax was right, so it is blind to confidence. The practical consequence is that the two metrics can disagree. Model A may get more languages right while stating 0.99 on everything, including the ones it gets wrong; model B may be right slightly less often but honest about its uncertainty. A wins on accuracy, B wins on log loss. Which you prefer depends on what happens downstream: if the 20-language identifier's output feeds a routing rule with an abstain threshold, or is combined with other evidence, the calibrated model is worth more than the marginally more accurate one. If a hard label is consumed and nothing else, accuracy may be the metric that matters. A related caution: log loss is an average over rows, so with a heavily skewed language mix it is dominated by whichever languages are common. It tells you nothing per class on its own. ## What to say in an interview Give the formula, say that only the true-class probability enters, anchor the number against `log(K)` and the class-prior baseline, and convert it back with `exp(-loss)` to make it concrete. Then name the two behaviours that make it useful and dangerous: it rewards honest calibration, and it is dominated by confident mistakes.