For a PyTorch MLP explained with shap, when do you choose DeepExplainer over GradientExplainer, and what does each cost per row?
answer
- both need the model object
- layer rules versus sampled gradients
- background size versus nsamples
- an unrecognised module only warns
- additivity is checked on one side
basics
~20 sChoose DeepExplainer when every layer is in its rule table: one backward pass over the background per row, additivity checked. Choose GradientExplainer for layers it cannot handle: 200 sampled gradient evaluations per row by default, with sampling noise.
solid answer
~50 sBoth take the network itself, `shap.DeepExplainer(model, data)` and `shap.GradientExplainer(model, data, batch_size=50, local_smoothing=0)`, with a background of tensors, and both detect PyTorch from the model object. `DeepExplainer` attaches DeepLIFT-style rules to known `nn.Module` layers (Linear, convolutions, batch norm, ReLU, Sigmoid, Tanh, Softmax, GELU, max-pooling and more). For each explained row it tiles the row against every background row and runs one backward pass, so cost grows with the background, and `shap_values` checks additivity by default, raising an assertion when the sum misses the model output by more than 0.01. A module it does not recognise only warns, "unrecognized nn.Module", and passes the gradient through. `GradientExplainer` computes expected gradients: `nsamples=200` random background-and-interpolation points per row, in batches of `batch_size`, so cost tracks `nsamples` rather than background size; it has no additivity check, and its values vary with the seed (`rseed`).
go deeper
Recall that DeepExplainer and GradientExplainer both need the network object and a background of tensors, and that model-agnostic explainers are the fallback when you only have a function.
Explain the cost drivers: DeepExplainer scales with background rows and checks additivity, GradientExplainer scales with nsamples and returns sampled, slightly noisy values.
Show you would audit layer support before trusting DeepExplainer, treat an additivity failure as a signal rather than a flag to disable, and seed GradientExplainer for reproducible batch output.
Decide when a team standardises on one network explainer, weighing deterministic self-checked output against coverage of custom layers, and how that choice constrains model architecture reviews.
## Two calls for a network When the model is a differentiable network, the **shap** library can use its gradients instead of calling it thousands of times as a black box. It offers two classes for that, and the choice between them is about **which layers the network contains**, **what each row costs**, and **how exact the output has to be**. Take a freight brokerage's small PyTorch MLP that reranks candidate carriers for each quote. Both constructors detect the framework by asking the model for `named_parameters()`, so they need the `nn.Module` itself (or a `(model, layer)` tuple to attribute a layer's input). A NumPy `predict` wrapper hides the gradients, and then only the model-agnostic explainers apply. How DeepLIFT-style rules or integrated gradients work as attribution methods is the subject of gradient-attribution material; here they are library calls with costs. ## DeepExplainer `shap.DeepExplainer(model, data)` takes the network and a background of tensors. - **Rules per layer.** It registers hooks on the network's `nn.Module` layers and applies a rule chosen by layer type: linear layers, convolutions, batch norm and average pooling are handled as linear, ReLU, ELU, Sigmoid, Tanh, Softplus, Softmax, SELU and GELU as nonlinearities, max-pooling with its own rule, dropout, flatten and identity passed through. - **Unknown layers.** A module type outside that table produces a warning, "unrecognized nn.Module: <name>", and its gradient is passed through unchanged. Because the rules hook modules, an operation written as a plain function call inside `forward()` has no module for a rule to attach to either. - **Cost.** For each explained row and output, it tiles the row once per background row, concatenates the background, and runs one backward pass: cost grows **linearly with the background**. The docstring suggests about 100 to 1,000 random background samples, never the whole training set. - **Additivity check.** `shap_values(X, ranked_outputs=None, output_rank_order="max", check_additivity=True)` asserts that the attributions plus the expected value match the model output within 0.01, with the message "The SHAP explanations do not sum up to the model's output!" An unsupported operation is the usual cause. ## GradientExplainer `shap.GradientExplainer(model, data, session=None, batch_size=50, local_smoothing=0)` computes **expected gradients**: 1. for each explained row, draw `nsamples` (default 200, on the call or `shap_values`) pairs of a random background row and a random point between it and the row; 2. take the model's gradient at each point, in batches of `batch_size`; 3. average gradient times the difference from the background row. Consequences: - **Cost** tracks `nsamples`, not the background size: 200 gradient evaluations per row in 4 batches of 50 by default. - **No additivity check.** The class only promises that the values converge towards summing to the output minus its expected value as sampling grows. - **Noise.** Two runs differ unless you pass `rseed`; `return_variances=True` returns variance estimates alongside the values. `local_smoothing` adds Gaussian noise of that standard deviation to the input before sampling. - **No layer table.** It uses autograd, so custom modules and functional operations are fine. ## Choosing for the reranker | Question | DeepExplainer | GradientExplainer | |---|---|---| | Layers all in the rule table? | required for trustworthy values | not needed | | Per-row cost driven by | background rows | `nsamples` (200) | | Additivity checked? | yes, by default | no | | Run-to-run variation | none for a fixed background | yes, unless `rseed` is set | | Attribute a layer's input | `(model, layer)` tuple | `(model, layer)` tuple | An MLP built from `nn.Linear` and `nn.ReLU` modules is DeepExplainer's home ground: fast, deterministic for a fixed background, and self-checking. Reach for GradientExplainer when the network uses modules or functional operations DeepExplainer warns about or fails the additivity check on, or when an attribution at an internal layer of a larger network matters more than exact additivity. Either way, keep the warnings visible in a batch job: a silenced "unrecognized nn.Module" warning is how a wrong explanation reaches a report. ## Sizing the run - **DeepExplainer**: per explained row and output, roughly one forward and backward pass over twice the background size (the tiled row plus the background), so 100 background rows means batches of 200. - **GradientExplainer**: per explained row, `nsamples` points regardless of background size; halving `nsamples` halves the cost and raises the variance. - For both, `ranked_outputs` limits a multi-output network to its top outputs, which cuts work when only the leading score matters.
- DeepExplainer fails its additivity assertion on the reranker; what do you check before switching explainers?Look for "unrecognized nn.Module" warnings and for nonlinearities written as plain function calls inside `forward()`, since the rules attach only to known module types. Replacing a functional activation with its `nn` module often fixes it. If the network genuinely needs an unsupported operation, `GradientExplainer` avoids the rule table, at the price of sampling noise and no built-in additivity check. Turning `check_additivity` off only hides the mismatch.
- Can either explainer take the reranker's NumPy predict function instead of the module?No. Both find the framework by asking the model object for its parameters and then run backward passes, so they need the PyTorch `nn.Module` (or a `(model, layer)` tuple) and tensor inputs. A NumPy wrapper is a black box to shap, which leaves the model-agnostic explainers and their per-row cost in masked evaluations times background rows.
- DeepExplainer works on any PyTorch layer because it just uses autograd.
- GradientExplainer's cost per row grows with the number of background rows.
- GradientExplainer gives the same values on every run without setting a seed.
- Setting check_additivity=False fixes a DeepExplainer additivity failure.
- Either explainer can be given a NumPy predict function that wraps the network.