In the shap library, which explainer fits a LightGBM regressor, a PyTorch network and an opaque predict function, and why?
answer
- what can the explainer see inside?
- fitted trees can be walked directly
- networks expose their gradients
- a bare function only answers calls
- agnostic cost: evaluations times background rows
basics
~20 sUse TreeExplainer for the LightGBM model, DeepExplainer or GradientExplainer for the PyTorch network, and PermutationExplainer or KernelExplainer for the opaque function: the more of the model an explainer can read, the faster and more exact it is.
solid answer
~50 sshap groups its explainers by how much of the model they can see. `TreeExplainer` reads the fitted trees of LightGBM, XGBoost, CatBoost and most scikit-learn tree models and runs Tree SHAP, which its docstring calls fast and exact, so explaining tens of thousands of rows is routine. For a network, `DeepExplainer` (DeepLIFT-style rules integrated over a background) or `GradientExplainer` (expected gradients) uses the network's own backward pass, and both need a PyTorch or TensorFlow model object. A function you can only call gets a model-agnostic explainer such as `PermutationExplainer`, which masks features, fills them from background rows and calls the function again and again, so its cost is masked evaluations per row times background rows. Every one of them except TreeExplainer's background-free path is handed a background dataset that stands in for a feature being unknown.
code
python · 15 linesimport shap
import torch
# LightGBM regressor: read the fitted trees directly
tree_explainer = shap.TreeExplainer(quote_model, data=X_background)
tree_sv = tree_explainer(X_quotes)
# PyTorch reranker: backward passes averaged over a tensor background
bg = torch.tensor(carrier_background, dtype=torch.float32)
deep_explainer = shap.DeepExplainer(reranker, bg)
deep_sv = deep_explainer.shap_values(torch.tensor(carrier_rows, dtype=torch.float32))
# Partner risk score: only a function, so mask features and call it
perm_explainer = shap.PermutationExplainer(partner_predict, X_background)
perm_sv = perm_explainer(X_quotes[:1000])go deeper
Recall the three families: TreeExplainer for supported tree ensembles, DeepExplainer or GradientExplainer for PyTorch and TensorFlow networks, and Permutation or Kernel explainers for anything you can only call.
Explain why access sets the cost: Tree SHAP walks the trees in compiled code, while model-agnostic explainers pay masked evaluations per row times background rows, and passing model.predict silently loses the tree path.
Show you would pick per model in a mixed system, keep the network as a model object so gradients stay available, and budget the black-box explainer separately because it dominates the run time.
Frame the trade as access against cost and exactness: getting a partner to expose a model object, or accepting sampled approximations, is a contract decision as much as a library choice.
## Why the explainer depends on what you can see The **shap** library computes attributions for one prediction: how much each input feature pushed the output away from a reference level. How it computes them depends on how much of the model it is allowed to inspect. There are three broad situations, and they map to three families of explainer: - **You hold a supported tree model object** (LightGBM, XGBoost, CatBoost, most scikit-learn tree ensembles): shap can walk every tree. - **You hold a differentiable network** (a PyTorch `nn.Module` or a TensorFlow / Keras model): shap can run backward passes through it. - **You hold only a function** that takes rows and returns predictions: shap can only change inputs and call it again. A freight quote engine is a good test case: a LightGBM regressor prices each load, a small PyTorch MLP reranks candidate carriers, and a partner's risk score is reachable only as a Python `predict` function. Each one gets a different explainer. ## Tree ensembles: TreeExplainer `shap.TreeExplainer(model, data=None, model_output="raw", feature_perturbation="auto")` parses the fitted trees and runs **Tree SHAP**, which the class docstring describes as a fast and exact method for tree models. It calls compiled code, either inside the model package or shap's own C extension, so it does not need to call `predict` thousands of times per row. That is why it is the first choice for the LightGBM quote model, and why sending a tree model anywhere else throws away both speed and exactness. Two details matter in practice: 1. Pass the **model object**, not `model.predict`. A bound method is not a tree model, so the unified `shap.Explainer` cannot recognise it and falls back to a model-agnostic algorithm. 2. A background is **optional** here. Without one, TreeExplainer uses the training counts stored in the trees; with one, it integrates over the background rows instead. ## Networks: DeepExplainer and GradientExplainer For the reranker, shap offers two calls that use the network's own gradients: - `shap.DeepExplainer(model, data)` applies DeepLIFT-style rules to each supported layer and averages over the background tensors. Its cost grows linearly with the number of background samples; the docstring suggests 100 to 1,000. - `shap.GradientExplainer(model, data, batch_size=50, local_smoothing=0)` computes **expected gradients**, sampling a background row and an interpolation point many times per explained row (`nsamples=200` on the call). Both detect the framework from the model object, so they need the PyTorch or TensorFlow model itself. A NumPy `predict` wrapper around the network hides the gradients and leaves you in the model-agnostic family. How these gradient methods differ as ideas belongs to the material on gradient attribution; the library question is only which call fits the model. ## Anything callable: the model-agnostic explainers The partner's score is a black box, so shap must **mask** features: replace some of them with values taken from background rows, call the function, and compare. The model-agnostic explainers are: | Explainer | What it does | Typical use | |---|---|---| | `PermutationExplainer` | walks random feature orderings forward and backward; call default `max_evals=500` | the general default for a callable with many features | | `ExactExplainer` | evaluates all 2^M masks for M varying features (with an `Independent` masker) | only a handful of features | | `PartitionExplainer` | follows a feature hierarchy from a clustering masker, giving Owen values | correlated feature groups, text, images | | `KernelExplainer` | the older standalone path: weighted regression over sampled coalitions | legacy code, or when you need its specific options | All of them pay for the same thing: **model calls**. Each explained row costs a number of masked evaluations, and each masked evaluation runs the function on up to one copy of the row per background row. That is the reason a model-agnostic run is slow, and why the background is usually cut to around 100 rows. ## The background every explainer needs Apart from TreeExplainer's background-free path, every explainer is handed **background data**: real rows that supply the values a "hidden" feature takes. Which rows to use, and why different backgrounds give different attributions, is a modelling question covered by the material on Shapley values. The library-level facts are simpler: the background is an argument (`data` or the masker), its size drives the cost, and shap's tabular maskers cut a large one down to `max_samples=100` rows unless told otherwise. ## Putting it together For the quote engine: - **LightGBM quote model**: `TreeExplainer`, with or without a background. - **PyTorch reranker**: `DeepExplainer` or `GradientExplainer`, given the `nn.Module` and a tensor background. - **Partner risk score**: `PermutationExplainer` (or `shap.Explainer`, which picks permutation for a 48-feature callable), budgeted carefully because it is by far the most expensive of the three.
- Why not send all three models through KernelExplainer, since it accepts any function?Because it only sees predictions. For each explained row `KernelExplainer` evaluates `nsamples` coalitions ("auto" is `2 * features + 2048`, so 2,144 for 48 features), each one over every background row, and it does not cut a large background down itself. On the LightGBM model that replaces TreeExplainer's fast exact values with a slower estimate, and on the network it ignores the gradients that make Deep and Gradient explainers cheap.
- What happens if you pass `quote_model.predict` instead of `quote_model` to `shap.Explainer`?The bound method is not a tree model object, so TreeExplainer's support check fails and `algorithm="auto"` treats it as an ordinary callable. With a 48-column background wrapped as an `Independent` masker, more than 10 features means the permutation algorithm. The run still works, just far slower and approximate, and nothing warns you. Pass the model object to keep the tree path.
- KernelExplainer is the right default for every model because it is model-agnostic.
- TreeExplainer can explain any model as long as you give it a background dataset.
- DeepExplainer can explain a NumPy predict function that wraps the network.
- A model-agnostic explainer is about as fast as TreeExplainer on a tree ensemble.
- Background data is only needed when explaining neural networks.