skip to content

An overnight shap job must explain 50,000 quotes from a 48-feature black-box predict function; how do you estimate its cost and cut it?

level: seniorimportance: should knowfreq 8%

answer

  1. count predictions, not explained rows
  2. masked evaluations times background rows
  3. one permutation costs 2M + 1
  4. max_evals 500 versus "auto"
  5. Kernel's nsamples and l1_reg defaults

basics

~20 s

Count model predictions: rows times masked evaluations per row times background rows. With shap 0.52's PermutationExplainer defaults that is 50,000 x 485 x 100, about 2.4 billion predictions; cut the background, the evaluation budget or the rows explained.

solid answer

~50 s

The unit of cost is a model prediction. A model-agnostic shap explainer spends a number of masked evaluations per explained row, and each one runs the function over the kept background rows. For 48 features, `shap.Explainer` builds a `PermutationExplainer`; one forward-and-backward permutation costs 2 x 48 + 1 = 97 evaluations, so the call default `max_evals=500` buys 5 permutations, 485 evaluations. With the masker's 100-row background that is 48,500 predictions per quote and about 2.4 billion for 50,000 quotes. Divide by the function's measured throughput and you have the run time. The levers, in order: a smaller background (`max_samples`), a smaller `max_evals` (noisier values, never below 97), a larger `batch_size` so each call scores more rows, fewer quotes explained, and parallel workers. Exact is out at 2^48 evaluations, and `KernelExplainer` at its defaults costs more and, in shap 0.52, keeps at most 10 non-zero attributions.

code

python · 10 lines
python
import shap

M, rows = 48, 50_000
evals = (500 // (2 * M + 1)) * (2 * M + 1)        # 485
print(rows * evals * 100)                       # 2,425,000,000 predictions

masker = shap.maskers.Independent(X_bg, max_samples=50)
explainer = shap.PermutationExplainer(partner_predict, masker, seed=0)
sv = explainer(X_quotes, max_evals=2 * M + 1, batch_size=100)
# 97 evaluations x 50 background rows = 4,850 predictions per quote

go deeper

for a junior

Remember that a black-box explainer calls the model many times per explained row, and that the background size multiplies that count.

for a middle

Be able to compute evaluations per row for Permutation (2M + 1 per permutation), Kernel (2M + 2048) and Exact (2^M), then multiply by background rows and rows explained.

for a senior

Turn the arithmetic into a plan: measure throughput, size the background and max_evals against noise, batch the calls, and decide which quotes really need an explanation.

for a principal

Set the budget at the system level: which decisions need per-row explanations, what noise is acceptable, and whether a partner should expose a model shap can explain natively.

## The unit of cost For a model you can only call, every shap explainer pays in **model predictions**. The formula for a batch is: **predictions = rows explained x masked evaluations per row x background rows per evaluation** A *masked evaluation* is one pattern of hidden and visible features. The masker turns it into one copy of the row per background row, with hidden features filled from that background row, and the function scores all of them. (The `Independent` masker skips copies that would not change, so treat the formula as an upper bound.) The scenario: a freight brokerage wants an explanation for each of 50,000 quotes overnight, and one of the scores is a partner's risk model reachable only as a Python `predict` function over 48 features. ## PermutationExplainer: the default for this case `shap.Explainer(partner_predict, X_bg)` wraps the DataFrame as an `Independent` masker, and with more than 10 features `algorithm="auto"` builds a `PermutationExplainer`. Its arithmetic in shap 0.52: 1. One permutation walks the features forward and backward: **2M + 1** evaluations, so 2 x 48 + 1 = **97**. 2. The call default `max_evals=500` buys 500 // 97 = **5** permutations, **485** evaluations. 3. The masker keeps **100** background rows by default, so one quote costs 485 x 100 = **48,500** predictions. 4. 50,000 quotes cost 48,500 x 50,000 = **2,425,000,000** predictions. `max_evals="auto"` means 10 x 2 x 48 = 960, which buys 9 permutations (873 evaluations) and 87,300 predictions per quote, about 4.4 billion in total. A `max_evals` below 97 raises a `ValueError`, because not even one permutation fits. Now measure. Time the function on a 10,000-row batch; if it scores 200,000 rows a second, 2.425 billion predictions take about 12,100 seconds, roughly 3.4 hours on one worker. Measured throughput, not a guess, decides whether the job fits the night. ## How the other explainers compare | Explainer | Evaluations per quote (48 features) | Predictions per quote, 100 background rows | |---|---|---| | `PermutationExplainer`, `max_evals=500` | 485 | 48,500 | | `PermutationExplainer`, `max_evals="auto"` | 873 | 87,300 | | `KernelExplainer`, `nsamples="auto"` | 2 x 48 + 2048 = 2,144 | 214,400 | | `ExactExplainer` | 2^48, about 2.8 x 10^14 | not feasible | `KernelExplainer` has two further catches in shap 0.52. It never sub-samples its data, so an unsummarised 20,000-row background would mean 2,144 x 20,000 = 42.88 million predictions per quote. And its `l1_reg` default is "num_features(10)" (since 0.47; it was "auto"), which selects at most 10 features per row and leaves the rest at zero. With `shap.kmeans(X_bg, 10)` its cost drops to 21,440 per quote, cheaper than the permutation default, but with ten weighted centroids standing in for the background and the 10-feature cap still on unless you pass `l1_reg=False`. ## The levers, in the order to pull them - **Background size.** Cost is linear in it. `shap.maskers.Independent(X_bg, max_samples=50)` halves the run; check on a few hundred quotes that the attributions you report barely move. - **Evaluation budget.** `max_evals=97` is one permutation: 9,700 predictions per quote, about 485 million in total, with noisier values. Set `seed=` on the explainer so reruns are reproducible. - **Batch size.** `batch_size` ("auto" means 10 masked evaluations per call for a tabular masker) does not change the prediction count, but larger batches amortise per-call overhead when the function is a remote or vectorised scorer. - **Rows explained.** Explain the quotes someone will read, for example declined or repriced ones, and a random sample for monitoring. - **Parallelism.** Rows are independent, so split the 50,000 quotes across workers. ## Common estimation mistakes - Counting **rows explained** as the cost, and forgetting that each one is hundreds of masked evaluations. - Forgetting the **background multiplier**, or assuming it is the full table when the masker has cut it to 100 rows (or, for `KernelExplainer`, assuming it was cut when it was not). - Reading `max_evals` as the number of permutations rather than the evaluation budget. - Timing the function on single rows, when shap calls it with batches of masked copies. ## Write the estimate down A batch explanation job should log the explainer class, `max_evals`, background size and measured predictions per second, and compare them against the night's window before it starts. The arithmetic is cheap; discovering at 6 a.m. that the job is a third done is not.

  • Why does max_evals="auto" give 9 permutations for 48 features rather than 10?
    "auto" sets `max_evals` to 10 x 2 x 48 = 960, but one forward-and-backward permutation costs 2 x 48 + 1 = 97 masked evaluations, not 96. `PermutationExplainer` runs `max_evals // 97` permutations, and 960 // 97 is 9. The extra evaluation per permutation is the all-hidden starting point, so the "10 permutations" reading is off by one permutation at this width.
  • Someone proposes KernelExplainer with nsamples=500 to save time; what do you check first?
    That the comparison is fair and the output is complete. At 500 coalitions and 100 background rows it costs 50,000 predictions per quote, similar to the permutation default, and it never sub-samples a large background on its own. In shap 0.52 its default `l1_reg="num_features(10)"` also leaves at most 10 of 48 features non-zero, so you would pass `l1_reg=False` if every feature must be attributed. Note that `nsamples` goes to `shap_values`, not to the call.

  • The cost is just the number of rows explained times one model call each.
  • Background size changes accuracy but not the run time of a model-agnostic explainer.
  • max_evals="auto" always means exactly 10 permutations.
  • ExactExplainer is fine for 48 features if you give it enough time overnight.
  • KernelExplainer at its defaults attributes all 48 features of every quote.

Assumes shap 0.52: PermutationExplainer's max_evals=500 call default and the 100-row masker ceiling are read from the 0.52 source, and KernelExplainer's l1_reg default became "num_features(10)" in 0.47.