How does `shap.plots.waterfall` build one patient's readmission explanation from the Explanation's `.base_values`, `.values` and `.data`?
answer
- start at the average output
- one bar per feature, largest on top
- base plus bars lands on f(x)
- max_display=10 and the leftover bar
- one row in, never the whole batch
basics
~20 sA shap waterfall takes one row of an Explanation: it starts at base_values, the average model output E[f(X)], adds each feature's SHAP value from .values as a bar labelled with its .data value, and ends at that patient's output f(x).
solid answer
~40 s`shap.plots.waterfall(sv[i])` explains one prediction. The bottom tick, `E[f(X)]`, is the row's `.base_values`: the model's average output over the reference data the explainer used. Each feature then gets a bar of length `.values[j]`, red pushing the output up and blue pushing it down, sorted by magnitude and labelled `value = name` from `.data` (or `.display_data` when it is set). Because the values are locally additive, `base_values + values.sum()` lands exactly on the top tick, `f(x)`. With the default `max_display=10`, the nine largest features get their own bars and the tenth bar collapses the rest, so a 30-feature model shows "21 other features". For a readmission model explained on its log-odds margin, I read every bar in log-odds, not in percentage points of risk.
code
python · 5 linessv = explainer(X_discharges) # Explanation for every discharged patient
row = sv[17] # one patient: 1-D Explanation
print(row.base_values) # E[f(X)], shared by every row
print(row.base_values + row.values.sum()) # f(x) on the explained scale
shap.plots.waterfall(row) # max_display=10: 9 bars + "21 other features"go deeper
Recall the three ingredients: base value at the bottom, one signed bar per feature from .values, and f(x) at the top, with the feature's actual value printed beside each bar.
Explain the additivity that makes the plot close and the max_display cut: nine named bars plus a grouped bar by default, and why the axis is the model's explained scale.
Show you can hand a waterfall to non-specialists safely: label the scale, explain what the base value is averaged over, and refuse causal or global readings of one patient's bars.
Weigh what a team standardises before per-patient plots leave the data team: one explainer and reference set per model version, a fixed scale and a stated max_display.
## What one row of an Explanation holds Calling a shap explainer on a batch returns a `shap.Explanation`. Indexing it with one row number, `sv[17]`, gives the explanation of a single prediction, and a waterfall plot is a picture of exactly that object. The attributes it reads are: | Attribute | For one row it holds | Where it shows up in the waterfall | |---|---|---| | `.base_values` | One number: the model's expected output over the explainer's reference data | The bottom tick, `E[f(X)]` | | `.values` | One SHAP value per feature, on the model's explained scale | The length and direction of each bar | | `.data` | The patient's raw feature values | The `value = name` label beside each bar | | `.display_data` | Optional human-readable values (category names, unscaled units) | Used for the labels instead of `.data` when set | | `.feature_names` | Column names | The feature names in the labels | The **base value** is not the patient's personal starting risk. It is the average output the model gives across the reference distribution: the background the explainer was handed, or, for a tree model explained without one, the training data recorded in the trees. Every patient explained by the same explainer shares it. ## How the waterfall is drawn `shap.plots.waterfall(shap_values, max_display=10, show=True)` builds the picture in four steps: 1. It checks that it was handed a **single** explanation. A matrix of rows raises a `ValueError` that suggests `shap.plots.waterfall(shap_values[0])`. 2. It sorts features by the absolute size of their SHAP values, largest at the top. 3. Starting from `E[f(X)]` at the bottom, it stacks one bar per feature: **red** bars push the output up, **blue** bars push it down. 4. It finishes at `f(x)` at the top, the model's output for this patient on the explained scale. When the model has more features than `max_display`, only `max_display - 1` features get their own bar. The last bar is labelled "N other features" and carries the summed SHAP values of everything not shown, so the arithmetic still closes. With 30 features and the default of 10, that is nine named bars plus "21 other features". ## A worked readmission example Take a hospital's 30-day readmission classifier, an XGBoost binary model explained on its raw log-odds margin, and one discharged patient: - `E[f(X)] = -1.20`, the average margin across the reference patients - `prior_admissions_12m = 3` pushes up by `+0.62` - `length_of_stay = 9` pushes up by `+0.31` - `age = 78` pushes up by `+0.18` - `follow_up_booked = 1` pushes down by `-0.45` - the remaining features, shown and grouped, sum to `-0.31` The bars sum to `+0.35`, so `f(x) = -1.20 + 0.35 = -0.85`. That is a **margin**, not a probability: passed through the logistic function it is about 0.30, against about 0.23 for the base value. A care-management reader must be told the axis is log-odds, or they will read "+0.62" as "62 points of risk". ```python sv = explainer(X_discharges) # Explanation, shape (n_patients, 30) row = sv[17] # one patient print(row.base_values + row.values.sum()) # equals f(x) on the explained scale shap.plots.waterfall(row) # 9 named bars + "21 other features" shap.plots.waterfall(row, max_display=15) # 14 named bars + "16 other features" ``` ## Common misreadings - **Treating bars as global importance.** A bar is this patient's contribution only; another patient with the same feature value can get a different bar because the other features differ. - **Expecting the bars to sum to the predicted probability.** They sum to the model's output on the explained scale, which for this model is the margin. - **Reading "other features" as ignored features.** It is the summed contribution of everything below the cut, and it can be large when many small effects point the same way. - **Reading a bar as causal.** A bar says how the model's output moved, not what would happen to the patient if the value changed. ## Neighbouring one-row views - `shap.plots.force(sv[17])` draws the same additive story as a horizontal tug-of-war. It renders through JavaScript, so a notebook needs `shap.initjs()` first; without it the cell shows "Visualization omitted, Javascript library not loaded!". `matplotlib=True` draws a static version for a single row. - `shap.plots.bar(sv[17])` draws the same row as signed bars without the running total. The waterfall stays the usual choice for a per-patient handout because it shows the starting point, every step and the end point on one axis.
- Why does `shap.plots.waterfall(sv)` fail when `sv` holds the whole batch?The waterfall plots a single explanation. A 2-D Explanation of rows by features raises a `ValueError` saying it can only plot one explanation and suggesting `shap.plots.waterfall(shap_values[0])`. Slice the patient you want, `sv[17]`, or use a many-row plot such as `shap.plots.beeswarm(sv)` for the whole batch.
- What does the bottom "21 other features" bar contain, and when should you change `max_display`?It is the sum of the SHAP values of every feature below the cut, so the plot still adds up to `f(x)`. With 30 features and `max_display=10`, nine features are named and 21 are grouped. If that bar is as long as the named ones, many small effects share the push, and raising `max_display` shows them.
- How do you show the same patient as a force plot in a notebook?Run `shap.initjs()` once, then `shap.plots.force(sv[17])`. The force plot renders through JavaScript; without `initjs()` the cell prints "Visualization omitted, Javascript library not loaded!". For a static image, `matplotlib=True` works for a single row but raises `NotImplementedError` for several rows.
A waterfall reads like a bank statement: the opening balance is the base value, each line is a deposit or withdrawal, small items are lumped into one sundry line, and the closing balance is the opening balance plus every line, all in the account's own currency, which here is log-odds.
- The base value is this patient's own risk before any features are considered.
- The bars of a waterfall add up to the predicted probability for every classifier.
- The other-features bar holds features the model ignored.
- A waterfall bar shows how important the feature is across all patients.
- You can pass the whole batch to the waterfall to see every patient at once.