Robust and Quantile Losses

## The loss Pinball loss (also the check loss, or quantile loss) is defined for a chosen level `tau` between 0 and 1. With actual value `y` and prediction `q`: ``` L(y, q) = tau * (y - q) if y >= q (under-predicted) L(y, q) = (1 - tau) * (q - y) if y < q (over-predicted) ``` Both branches are non-negative and both are zero when `q = y`. The loss is piecewise linear with a kink at zero -- the shape gives it its name, the two straight arms hinged like a pinball flipper. The only thing that distinguishes it from absolute error is that the two arms have different slopes: `tau` on the under-prediction side, `1 - tau` on the over-prediction side. At `tau = 0.5` the slopes are equal at one half each, so pinball loss becomes half the absolute error and the fitted value is the median. Every other tau tilts the flipper. ## Why the optimum is the tau-quantile Take a single number `q` predicted for a whole distribution. Expected loss is ``` E[L] = tau * E[(y - q) when y > q] + (1 - tau) * E[(q - y) when y < q] ``` Nudge `q` upward by a tiny amount. Every outcome below `q` gets worse, at rate `(1 - tau)` each, and there is a `P(y < q)` share of them. Every outcome above `q` gets better, at rate `tau` each, and there is a `1 - P(y < q)` share of them. The derivative is therefore ``` (1 - tau) * P(y < q) - tau * (1 - P(y < q)) ``` Setting it to zero and solving gives `P(y < q) = tau`. The minimiser is precisely the value the target falls below with probability tau -- the tau-quantile. With features in the model, the same argument applies pointwise and the fit estimates the **conditional** tau-quantile. Notice what the argument used: only the *probability* on each side of the prediction, never how far away the outcomes on the far side were. That is why quantile fits inherit absolute error's insensitivity to how extreme the tail values are. ## Reading the asymmetry as a promise A food-delivery ETA is a good illustration. The customer sees one number. Quoting the conditional mean means roughly half of deliveries arrive later than promised, which reads as unreliable however good the model's average error is. Quote instead the conditional 90th percentile, fitted with pinball loss at `tau = 0.9`: the quote is deliberately pessimistic, and by construction only about one delivery in ten runs past it. The model has not become more accurate; it has been asked a different question -- "what time will this beat nine times out of ten?" rather than "what is the expected time?". That also tells you how to sanity-check such a model in production: measure the share of actual outcomes that fall below the prediction on held-out data. For `tau = 0.9` it should sit near 90 percent. If it is 97 percent your quotes are needlessly padded; if it is 78 percent you are breaking a promise you advertised. ## Practical points - **One tau, one model.** A single quantile fit gives one slice of the conditional distribution. Wanting several quantiles means fitting several models, each with its own tau. - **Extreme taus are data-hungry.** At `tau = 0.99` the fit is pinned by the top one percent of rows, so the estimate is noisy unless the dataset is large. - **It is not differentiable at zero.** Like absolute error, the loss has a kink where the residual crosses zero, so it is fitted with methods that tolerate a subgradient, or with a smoothed approximation. - **The loss is also the evaluation metric.** Average pinball loss at the same tau on held-out data is the natural way to compare two candidate quantile models -- and the natural companion check is the empirical coverage described above, because a model can score reasonably on average pinball loss while being systematically mis-calibrated in one segment. - **The prediction moves, and everything downstream notices.** Swapping from a mean fit to a `tau = 0.9` fit lifts essentially every prediction. Anything that consumes those numbers -- capacity plans, promised times, alerting thresholds -- is now reading a different quantity, and should be told so. ## The common misreading Candidates often describe a `tau = 0.9` fit as "a model with a safety margin added". It is not an additive offset. The gap between the median fit and the 90th-percentile fit is whatever the local spread of the conditional distribution demands: wide during a chaotic Friday dinner rush, narrow on a quiet Tuesday morning. A quantile fit learns that varying width from the features; a constant margin bolted onto a mean fit cannot.