When are expected cell counts too small for a chi-square test, and what do you use instead?

## Why there is a rule at all The chi-square statistic is computed from counts, which are discrete, but it is compared against a continuous chi-square distribution. That comparison is justified by a limiting argument: as the expected count in each cell grows, the standardised cell deviations `(O - E)/sqrt(E)` behave increasingly like normal variables, and the sum of their squares behaves increasingly like a chi-square variable. When expectations are small the approximation is poor, and typically **anti-conservative** — the reported p-value is smaller than the truth, so you over-reject. ## The rules of thumb Two versions circulate and both are worth being able to state: - **The strict version:** every expected count is at least 5. - **Cochran's more permissive version:** no expected count is below 1, and at most about 20% of cells have expected counts below 5. Neither is a theorem. They are practical guardrails, and interviewers mainly want to see that you know the rule constrains **expected** counts. This is the single most common error: a candidate points at an observed zero and declares the test invalid. An observed zero in a cell that expected 40 is a striking finding, not a violation. Conversely a cell that observed 3 but expected 0.4 is exactly the problem case, whatever the observed number happens to be. ## What to do when it fails **Collapse categories.** If a survey has "fully on-site" answered by four people, folding it into an adjacent category raises the expectations. This must be defensible on substance, decided before you see the p-value; merging categories to manufacture significance is a form of data dredging. **Get more data.** Expected counts scale with N, so doubling the sample doubles every expectation. Often unavailable, but it is the honest first answer. **Compute an exact p-value.** Instead of leaning on the limiting distribution, enumerate the possible tables and add up the probabilities of those at least as extreme as the one observed. For a 2x2 table this is **Fisher's exact test**; for larger tables the same idea generalises through exact or permutation-based enumeration, at greater computational cost. ## Fisher's exact test Fisher's test conditions on **both** sets of margins being fixed at their observed values. Under that conditioning and the null hypothesis of no association, the count in any one cell follows a **hypergeometric distribution** — the same distribution as drawing without replacement from an urn. Because the margins pin down the whole table once one cell is known, you can enumerate every possible table, compute each one's probability exactly, and sum the probabilities of the tables at least as extreme as yours. No large-sample approximation is involved, which is why it remains valid however small the counts are. The design that motivated it is the **lady tasting tea**. A taster claims she can tell whether milk or tea was poured first. Eight cups are prepared, four each way, and she is told there are four of each and asked to identify which four had milk first. Under the null hypothesis that she is guessing, the number of correctly identified cups follows a hypergeometric distribution over the C(8,4) = 70 equally likely selections. Getting all four right happens with probability 1/70, about 0.014 — small enough to reject guessing at the 5% level, and only just. Getting three of four right is far more likely under guessing and would not be persuasive. This is also a neat illustration of the conditioning: the taster knows the margins (four and four), which is precisely what Fisher's test assumes. A nuance worth knowing: because it conditions on margins that were not truly fixed by the design in most observational tables, Fisher's test is often **conservative** — its actual rejection rate can sit below the nominal level. So "exact" means the p-value is computed exactly under its own conditional null, not that it is uniformly the most powerful choice. ## Related repairs to know about For 2x2 tables, **Yates' continuity correction** subtracts 0.5 from each absolute deviation before squaring, nudging the discrete statistic toward the continuous reference. It shrinks the statistic and enlarges the p-value, and it is widely considered over-conservative; most practitioners now prefer either the uncorrected statistic when expectations are adequate or an exact test when they are not. ## The judgment call Sparse tables are usually a signal about the study, not just the arithmetic. A cell expecting 0.3 means you have almost no information about that combination, and no test will conjure it. The right response is often to say so, present the raw counts, and either widen the categories or plan a larger collection — rather than to hunt for a method that returns a number.