skip to content

Model Interpretability

Explaining a fitted model: permutation importance and partial dependence globally, Shapley values and counterfactuals for one row, audits of who it fails. Interviewers make you defend a black box.

on this pageshow

explore

questions

page 2 of 2

How does reweighing training rows shrink a group disparity before any model is fitted?

level: middleimportance: nice to knowfreq 30%

basics

~20 s

Each training row gets a weight equal to the count its (group, outcome) cell would have if group and label were independent, divided by the count actually observed. The weighted data carries no group-label association, and scoring never needs the protected column.

open as a page

In a generalized additive model, how does a shape function keep a non-linear effect readable?

level: middleimportance: nice to knowfreq 26%

basics

~10 s

A generalized additive model fits one curve per feature and sums them, so a feature's whole effect is a single readable plot: non-linear in shape, never tangled with the other features.

open as a page

For an anchor rule explaining one model prediction, what do precision and coverage each mean?

level: middleimportance: nice to knowfreq 24%

basics

~20 s

Precision is the share of rows satisfying the rule that the model still labels the same way as the explained row; coverage is the share of the population that satisfies the rule at all. Higher precision usually buys narrower coverage.

open as a page

How do you rank adverse-action reason codes when correlated features split the attribution?

level: seniorimportance: nice to knowfreq 30%

basics

~20 s

Rank at the level of business reason codes, not raw features: map correlated inputs to one code and sum their contributions inside it. Add a deterministic tie-break so equal contributions produce the same order every run.

open as a page

Why prefer accumulated local effects over partial dependence when two input features are strongly correlated?

level: seniorimportance: nice to knowfreq 26%

basics

~20 s

Partial dependence sets one feature to each grid value in every row, manufacturing combinations the data never contains, such as a tiny engine in a very heavy car. Accumulated local effects compare predictions only within each row's own neighbourhood.

open as a page

Which group fairness criterion should a marketplace commit to for a fraud model that routes sellers into manual verification?

level: principalimportance: nice to knowfreq 26%

basics

~20 s

Start from the harm. Extra document checks burden a legitimate seller, so equalizing the false-positive rate across groups is the commitment to state publicly. Prefer equal flag rates instead when the fraud labels are themselves untrustworthy.

open as a page

Why can a health-risk model be biased when its label is prior healthcare spend?

level: principalimportance: nice to knowfreq 30%

basics

~20 s

Spend is a proxy for need, and the two come apart by group. A group with equal illness but less access spends less, so it scores as lower risk. The bias sits in the target definition, where no feature audit will find it.

open as a page

Permutation importance ranks a feature first — what business decisions does that actually license?

level: principalimportance: nice to knowfreq 32%

basics

~10 s

Only model-facing ones: monitoring priority, data-collection spend, sanity checks against domain expectation. It measures one model's reliance on a column under one dataset and metric, not evidence that changing the quantity would change outcomes.

open as a page

What would you require before per-decision explanations are shown to customers or regulators?

level: principalimportance: nice to knowfreq 24%

basics

~10 s

Treat the explanation as a product surface with its own bar: a faithfulness test the team runs, a measured stability budget across seeds and retrains, a stated purpose, and a fallback when checks fail.

open as a page

How does the choice of background set change a Shapley explanation of the same prediction?

level: principalimportance: nice to knowfreq 28%

basics

~20 s

The background set defines what the prediction is compared against, so it decides the whole explanation: attributions sum to the prediction minus the background's prediction, and a feature already sitting at its background value gets zero credit.

open as a page

showing 31–40 of 40