In a residual-versus-fitted plot, what does a funnel widening with the predicted value mean?

## What the plot is For every evaluation row, compute the **residual** `r = actual - predicted` and plot it on the vertical axis against the model's **fitted value**, its own prediction, on the horizontal axis. The healthy picture is a formless horizontal band: residuals scattered symmetrically around zero, with about the same vertical spread everywhere across the range of predictions. Every interesting departure from that is a shape. This is really a continuous form of slicing. Binning predictions into deciles and reporting the metric per bin gives the same information in a table, which is what goes into a written evaluation; the plot is how you find it. ## The funnel A funnel — tight near the left, fanning out to the right — says the **magnitude of the error grows with the magnitude of the prediction**. The average residual can stay at zero throughout, so the model is not necessarily biased; its precision is simply not uniform. This is extremely common, and often it is the honest structure of the problem rather than a defect. Durations, counts, prices, demand and spend are positive, right-skewed quantities whose noise is naturally multiplicative: a long trip has more opportunities to go wrong, so being ten percent out on an hour is ten times as many minutes as being ten percent out on six minutes. Where the underlying process is count-like, the variance grows with the mean by construction. ## Why it changes how you read the headline Three consequences follow directly. 1. **The headline metric belongs to the big end.** If the largest-prediction decile carries most of the absolute error, an MAE improvement is mostly an improvement on that decile, and can coexist with the small-prediction rows getting worse. 2. **Training follows the same gradient.** A squared-error objective weights large residuals hardest, so the fit spends its capacity where the residuals are already largest. 3. **Absolute and relative accuracy disagree.** The model may be uniformly accurate in percentage terms and wildly non-uniform in minutes. Which one the user experiences depends on the decision downstream: a promise shown to a customer cares about minutes; a capacity plan may care about percentages. ## What to do about it - **Report by band, always.** Split predictions into bands or deciles and publish the count, the absolute error and the relative error for each. This is the leaf-level answer and it costs nothing. - **Match the metric to the decision.** If the cost of an error really is proportional to size, an absolute headline is the wrong summary and a relative one is closer. - **Consider modelling the target on a log scale**, or with a multiplicative form, when the error genuinely is proportional. This stabilises the spread rather than merely reporting it, but it changes what the model optimises, so verify on the original scale. - **Do nothing but document**, when the funnel is intrinsic and the bands are all acceptable. Not every pattern is a bug. ## Funnel versus curvature versus clumps The same plot shows three different diseases and they need distinguishing: - **Funnel**: mean residual near zero everywhere, spread increasing. A variance pattern; the model is less precise on big values. - **Curvature**: the mean residual is systematically positive in one part of the range and negative in another — for instance the model under-predicts the smallest and largest values and over-predicts the middle. That is **bias**: a missing nonlinearity, a missing interaction, or a model that is too constrained. It is the more serious finding, because it is a correctable and systematic error rather than an unavoidable spread. - **Clumps or stripes**: distinct blobs sitting off the main band usually mark a cohort — a product line, a region, a data-source — behaving differently. That is the cue to switch from the plot to a named-cohort table. ## Two traps **Plot against the prediction, not the actual value.** Residual `y - yhat` contains `y`, so plotting residuals against `y` produces an upward trend even when the model is unbiased. That trend is an artefact of the axis choice, not a finding. **Watch the floor.** If the target cannot be negative, the residual satisfies `r >= -predicted`, so at small predictions residuals simply cannot be very negative. That alone creates an asymmetric narrowing at the left that can look like a funnel. Check whether the spread on the positive side widens too before concluding anything. ## What interviewers listen for A correct definition of residual and fitted value; the ability to separate a variance pattern from a bias pattern in the same picture; the instinct to convert the plot into a banded error table that a stakeholder can read; and the judgement not to treat every funnel as a defect requiring a fix.