How does MASE scale forecast errors, and what does a MASE above 1 mean?

## The problem MASE is built for Two demands pull against each other when you score forecasts across a catalogue of series. You want a number in units so it means something operationally, and you want a number that is comparable between a SKU selling four units a week and one selling forty thousand. Absolute error is interpretable but not comparable. Percentage error is comparable but undefined near zero and unbounded. Mean absolute scaled error resolves this by comparing an error against another error rather than against the data. ## The definition With actuals `y_t` and forecasts `f_t` over a held-out window of h periods, and a training sample of n observations: ``` numerator = (1/h) * sum over the test window of |y_t - f_t| denominator = (1/(n - m)) * sum over t = m+1..n of |y_t - y_{t-m}| MASE = numerator / denominator ``` The denominator is the in-sample mean absolute error of the seasonal-naive rule, which forecasts each period as the value m periods before it. For monthly data with yearly seasonality m is 12; for daily data with a weekly cycle m is 7; for a series with no seasonality m is 1, which makes the rule the plain random-walk naive forecast. Three properties follow directly. The units cancel, so the number is dimensionless. It is finite for any series that is not perfectly periodic, because the denominator is zero only if the naive rule was exactly right at every training point. And it is symmetric in the sense that over- and under-forecasting of the same magnitude cost the same, unlike percentage-based measures. ## Reading the number MASE below 1 means the model's average absolute error on the test window is smaller than the naive rule's average absolute error on the training data. Below 1 is the bar; how far below tells you how much the model bought you. MASE of 0.6 is a meaningful gain. MASE of 0.95 on a model that costs a nightly retraining job is a conversation about whether to ship the naive rule instead. MASE above 1 means the opposite: the model averaged larger absolute errors than a rule that copies the value from one season ago. On a portfolio, the share of series with MASE above 1 is often more informative than the mean MASE, because it counts the series where you are actively losing. ## The subtlety worth raising The denominator is measured in-sample and one step ahead, while the numerator is measured out-of-sample and may cover many horizons. Those are not the same task. A test window that happens to be more volatile than the training period inflates the numerator without touching the denominator, and a multi-step forecast is intrinsically harder than a one-step one. So a MASE of 1.1 at horizon 12 does not literally prove that a seasonal-naive forecast would have beaten your model on that window — it says your model's twelve-step errors exceeded the naive rule's one-step errors during training. If you want the strict head-to-head, compute the benchmark's forecasts on the same held-out window at the same horizons and report a skill score, `1 - error_model / error_benchmark`, where positive means better. Some teams compute MASE with an out-of-sample denominator for exactly this reason; that is a defensible variant, but it is no longer the textbook definition, so say which one you used. Mixing the two across a report makes the numbers incomparable. ## Where it does and does not apply MASE is a point-forecast metric for a series with enough history to estimate the scaling constant. It needs more than m training observations, and it is unstable when the training sample is very short, because the denominator is then an average over a handful of differences. For a brand-new SKU with three weeks of history the scaling constant is noise, and a units-based error plus a stated benchmark is more honest. It also says nothing about uncertainty. A model can have an excellent MASE and prediction intervals that cover far less than their nominal rate, because MASE only ever looks at the point forecast. Distributional accuracy needs a separate score. Finally, because it is built on absolute error, MASE is minimised by forecasting the median of the predictive distribution. If the business cost of being short differs from the cost of being long, a metric aligned with the median is scoring the wrong target no matter how clean its scaling is. ## What a strong answer sounds like State the formula, name what the denominator actually is — in-sample, one-step, seasonal-naive — say that 1 is the break-even line, and then volunteer the caveat that the numerator and denominator are measured on different tasks. That last move is what separates a candidate who has read the definition from one who has reported the metric to a stakeholder.