When are excess zeros in a count model a reason to use a zero-inflated model?

## Two kinds of zero The whole idea rests on a distinction between two ways a zero can arise. - A **structural zero** comes from a unit that could not have produced an event at all. A user who has no payment method set up, is browsing for someone else, or is outside the product's addressable market records a zero for reasons the count process never touches. - A **sampling zero** comes from a unit fully capable of producing events that happened not to produce any in this window. A regular shopper who bought nothing in March is a sampling zero. If both are present, the observed pile of zeros is a mixture of two populations, and a single count model — which has only one story about how zeros arise — will under-predict them. ## What a zero-inflated model actually specifies A zero-inflated model is a mixture with two components: 1. A **binary component** giving the probability `p(x)` that a unit belongs to the always-zero group. It is usually modelled with its own predictors and its own link. 2. A **count component** (Poisson or negative binomial) generating counts for the remaining units with mean `mu(x)`. That component produces some zeros too. So the probability of observing zero is `p(x) + (1 - p(x)) * P(count component gives 0)`, and the probability of any positive value `k` is `(1 - p(x)) * P(count component gives k)`. A zero is therefore **ambiguous**: you never know which population it came from, only how likely each is. The interpretation follows the structure. Coefficients in the binary part describe who is likely to be a structural zero; coefficients in the count part describe the event rate among those who can have events. They are different questions with different signs and different audiences, and conflating them into 'the effect of X on purchases' is a classic reporting error. The same predictor can raise the chance of being a non-shopper while raising purchases among shoppers. ## Hurdle models: the cleaner alternative A **hurdle model** splits the outcome instead of mixing it: 1. A binary part models zero versus positive. 2. A **zero-truncated** count model handles the positive values only. Every zero comes from the first part; the count part cannot produce a zero at all. There is no ambiguity about a zero's origin, and the two parts map cleanly onto 'did anything happen' and 'given something happened, how much'. Hurdle models fit naturally when the first event is a genuinely different decision from the subsequent ones — signing up before purchasing, opening a ticket before filing more. Choose a zero-inflated model when the structural-zero population is a real, describable group; choose a hurdle model when the split is a decision rather than a population. ## The discipline: excess zeros are not evidence of inflation This is where most candidates go wrong. Count data with substantial variability produce a great many zeros without any mixture at all. A negative binomial fit, with its extra dispersion parameter, can reproduce a startling spike at zero as a natural consequence of heterogeneity in the rate. Zero inflation and overdispersion are different departures from the simple count model: one adds mass specifically at zero, the other adds variability across the whole range — and both inflate the observed count of zeros. So the question is never 'are there a lot of zeros' but 'does a simpler model already predict this many'. The practical check: fit the candidates, ask each how many zeros it expects, and compare against the number observed. If the negative binomial already lands close, the extra component buys complexity and nothing else. ## How to decide, in practice 1. **Substantive argument first.** Can you name the always-zero population and say why it exists? If the answer is hand-waving, stop. 2. **Compare predicted zero counts** from the plain count model, the overdispersed model, and the zero-inflated one against the observed count. 3. **Compare out-of-sample fit**, since the extra component adds parameters that can flatter in-sample measures. 4. **Check identification.** If the two components use identical predictors, they can be weakly separated and the inflation coefficients come back with huge standard errors — a sign the data are not really distinguishing two populations. A formal test comparing an inflated fit with its non-inflated counterpart is sometimes cited for this decision, but its validity in that setting is disputed, so lean on predicted-zero comparisons and out-of-sample performance instead. 5. **Consider zero-inflated negative binomial** when both problems are present: the positive counts can still be more variable than a simple count model allows even after the structural zeros are accounted for. ## What the choice costs A two-component model doubles the coefficient table, makes every summary statement conditional on which part you mean, and needs more data to estimate stably. That price is worth paying when the two populations are real and the distinction is the point of the analysis — telling a growth team how many users are simply not in the market is a different and often more valuable finding than a slightly better fit. It is not worth paying to make a histogram look tidier.