For an anchor rule explaining one model prediction, what do precision and coverage each mean?

## What an anchor is A local linear surrogate answers 'how much did each feature push?'. An **anchor** answers a different question: 'what set of conditions, once true, is enough to pin this prediction down?' It is an IF-THEN rule over feature predicates, and its promise is scope: as long as these conditions hold, the model's output is essentially fixed, whatever the other features do. For a resume-screening model that rejected one applicant, an anchor might read: `IF years_experience < 2 AND certification = none THEN reject (precision 0.96, coverage 0.04)` That single line carries two very different numbers, and interviewers ask about them precisely because candidates blur them. ## Precision **Precision** is the probability that the model gives the same prediction as it gave the explained row, *among inputs that satisfy the rule*. It is estimated by sampling: hold the anchor's conditions fixed at the explained row's values, let every other feature vary according to a perturbation distribution, score each sampled row with the model, and count how often the prediction matches. Precision 0.96 means 96% of those sampled rows were also rejected. Two qualifications matter. First, it is **probabilistic, not absolute**. Precision is estimated from a finite sample, so the search cannot certify it exactly; the guarantee is of the form 'the returned rule's true precision is at least the threshold with high probability'. Efficient anchor search is usually framed as a bandit problem — spend sampling budget on candidate rules that might beat the threshold, prune the ones that clearly cannot — because that is how you get a confident answer without scoring every candidate exhaustively. Second, precision is **relative to the perturbation distribution**. The number describes how the model behaves on the rows you sampled. If that sampler produces combinations that never occur in the applicant pool, the 0.96 is a fact about synthetic inputs and may not transfer to real ones. ## Coverage **Coverage** is the probability that a randomly drawn input satisfies the rule's conditions at all. It says nothing about correctness; it says how *broad* the explanation is. Coverage 0.04 means the rule speaks for about four percent of applicants. The reason coverage is reported is that precision alone can always be maximised trivially. Take the degenerate anchor that pins every feature of the row to its exact observed value: no other row satisfies it, the perturbation sample is the row itself, and precision is 1.0 by construction. That rule explains nothing — it is just the input restated. Coverage is what exposes it: near zero. ## The tradeoff Anchor search is typically greedy: start from the empty rule, add the condition that most improves precision, stop when precision clears the threshold. Each added condition can only shrink the set of inputs that satisfy the rule, so **precision climbs and coverage falls monotonically as the rule grows**. Raising the threshold from 0.90 to 0.99 therefore does not just make the rule stricter — it usually makes it longer and narrower, and a longer rule is also harder for a human to act on. A useful reading of the pair: - **High precision, decent coverage** — a genuinely informative rule; it describes a whole region of the model's behaviour. - **High precision, tiny coverage** — an honest explanation of this one case, but no basis for any general statement about the model. - **Modest precision** — the rule is not a sufficient condition; the model flips inside it, and showing it as an explanation misleads. ## Anchors versus a linear local surrogate The forms answer different questions and fail differently. A linear surrogate gives graded, signed contributions but leaves its region of validity implicit — you must decide separately how far it reaches. An anchor states its region explicitly, as conditions, with a measured precision attached, which makes it easier to hand to a non-technical reviewer: 'while these two things are true, the decision does not change.' The cost is that anchors need a threshold, a perturbation distribution and a sampling budget, they only apply where crisp predicates exist, and they say nothing about magnitude — no sense of how *strongly* a condition mattered, only that it sufficed. ## What to say in an interview Define both numbers precisely, name the degenerate maximum-precision rule as the reason coverage is always reported alongside it, and note that the precision is a sampled estimate with a confidence attached rather than a guarantee about every possible input.