Why prefer accumulated local effects over partial dependence when two input features are strongly correlated?

## The problem with sweeping one column Partial dependence is computed by pinning a feature at a grid value in **every** row and averaging the model's predictions. That is fine when the feature moves roughly independently of the others. It stops being fine when it does not. Take a fuel-economy model with **engine displacement** and **curb weight** as inputs. In the real fleet these move together: large engines sit in large, heavy vehicles. To draw the displacement curve, partial dependence still forces displacement to 6 litres in the row describing a 900 kg city car, and to 1.0 litres in the row describing a 2,300 kg pickup. Those rows are not in the data, are not on the road, and are places the model never learned anything about. Whatever the model happens to output there — the arbitrary extrapolation of a tree ensemble past its last split, say — is averaged into the plotted curve with full weight. The practical symptom is a displacement curve with a shape nobody can explain, driven by a region of input space that does not exist. ## Why the obvious fix is worse The intuitive repair is to stop making things up: at each value of displacement, average the predictions of only the rows that *already* have roughly that displacement. That conditional average is sometimes called an M-plot, and it introduces a different failure. Cars with 6-litre engines are also heavy, less aerodynamic and geared differently. Averaging their predictions gives you the combined behaviour of the whole correlated bundle, then labels the resulting curve "displacement". You have traded impossible inputs for misattributed credit. ## How accumulated local effects work ALE takes a third route: stay local, and measure **differences** rather than levels. 1. **Partition** the feature's range into intervals, usually by quantiles so each interval holds a similar number of rows. 2. **For each interval**, take the rows whose feature value falls inside it. For each such row, score it twice — once with the feature set to the interval's **lower** edge, once at its **upper** edge — keeping that row's other features untouched. Take the difference. 3. **Average** those differences within the interval. This is the local effect: how much the model's prediction moves when the feature nudges across this small interval, estimated only on rows that genuinely live there. 4. **Accumulate** the interval averages from left to right, a running sum, producing a curve. 5. **Centre** the accumulated curve so its mean over the data is zero. Because a row is only ever evaluated at the edges of the interval it already belongs to, the model is never asked about a 1.0-litre pickup. And because the quantity averaged is a *difference* computed within one row, everything else about that row cancels — the curve is not credited with the effect of curb weight the way a conditional average would be. ## Reading the result The y-axis is **relative**, not absolute. Because the curve was built by accumulating differences and then centred, a value of +2 mpg at some displacement means "about 2 mpg above the average prediction, attributable to this feature's accumulated local effect" — not "the model predicts 2 mpg here". Partial dependence, by contrast, is on the prediction's own scale. Comparing the two curves' heights directly is a beginner's error; compare their shapes. ## What ALE does not fix **It does not recover an independent effect that the data cannot identify.** If displacement and weight were perfectly collinear, no procedure could tell you what the model would do to a heavy car with a small engine, because nothing in the data ever showed it. ALE reports the effect of moving displacement *among the cars that actually have that displacement*, which is a meaningful and honest quantity, but it is a local, conditional one. **Interval width is a real knob.** Too few intervals over-smooths and can flatten genuine structure; too many leaves each local difference estimated from a handful of rows and the accumulated curve becomes visibly jagged. Quantile-based intervals with a floor on the row count per interval are the usual compromise. **It is still a model artefact.** Like any of these curves it describes what the model has learned from historical data, and it does not license a claim about what happens if you intervene on engine size. ## When to reach for which Check the correlation structure of the inputs first. With weakly correlated features the two curves agree closely and partial dependence is cheaper to compute and far easier to explain to a non-technical audience, so use it. When a feature is strongly tied to another, either plot the accumulated local effects instead, or plot partial dependence within strata where the correlation is broken. A candidate who reports both and shows they agree has done the cheap check that most people skip.