Why can cluster purity not be reported on its own as a clustering quality score?

## What purity actually computes Purity is an external validation measure: it scores a clustering against a set of known class labels. The recipe is: 1. Build the contingency table of clusters against true classes: cell `n_ij` is the number of points in cluster `i` that carry true class `j`. 2. For each cluster `i`, take `max_j n_ij` — the size of the single most common true class inside it. 3. Sum those maxima over all clusters and divide by `N`, the number of points. `purity = (1/N) * sum_i max_j n_ij` A tiny example: 10 points, true classes A A A A A A B B B B. A clustering produces cluster 1 = {A A A A B} and cluster 2 = {A A B B B}. Cluster 1's majority is A with 4; cluster 2's majority is B with 3. `purity = (4 + 3)/10 = 0.7`. The reading is intuitive — 70% of points sit in a cluster whose dominant label matches their own — and that intuitiveness is exactly why the measure keeps getting reported. ## The defect: monotone in the number of clusters Take any cluster and split it into two parts. Let the original cluster's majority class be `m` with count `c_m`, distributed as `c_m1` and `c_m2` across the two parts. The new contribution is `max_j n_1j + max_j n_2j`, and since `max_j n_1j >= c_m1` and `max_j n_2j >= c_m2`, the new contribution is at least `c_m1 + c_m2 = c_m`. Splitting therefore never decreases purity, and usually increases it. Carry that to the limit. With one cluster per point, every cluster's majority count is 1, the sum is `N`, and `purity = 1.0` — against any label set, however unrelated to the data. On a 5,000-row product taxonomy that means a clustering into 5,000 clusters scores a perfect 1.0 while telling you nothing. Long before the limit, the number is already inflated: a few hundred clusters over a few thousand rows will routinely land above 0.9. The opposite extreme is just as misleading in the other direction. With a single cluster covering everything, purity equals the proportion of the largest true class. On a label set where 80% of rows share one class, the do-nothing clustering already scores 0.8, so 0.8 is not a good score there — it is the floor. ## Two properties purity lacks **It has no completeness term.** Purity asks whether each cluster is internally homogeneous; it never asks whether a true class was kept together. Shattering one true class across 50 clusters costs nothing. Measures that count point *pairs* — the Rand family — do penalise this, because two points sharing a class but sitting in different clusters register as a disagreement. The information-theoretic family expresses the same idea as a pair of numbers, homogeneity and completeness, combined into V-measure. **It is not corrected for chance.** Purity has no baseline that random labelings score. Assign 1,000 points to 50 clusters at random and purity will still come out well above the largest-class share, purely because small random clusters are homogeneous by accident. Chance-corrected measures fix a reference point: the adjusted Rand index and adjusted mutual information both sit near 0 for independent labelings, so a score is interpretable without knowing k. ## When purity is still legitimate Purity is not banned; it is unsafe *unqualified*. It is fine for: - Comparing two clusterings that produce the **same** number of clusters — the inflation is then a constant across both. - Explaining a result to a non-technical audience, alongside a defensible measure, because the sentence 'x% of rows fall in a cluster dominated by their own category' lands where 'ARI = 0.31' does not. - Diagnosing which specific clusters are mixed, by reading per-cluster purity off the contingency table rather than the single aggregate. The practical rule for an interview: never quote a purity number without stating k, and never use purity to choose between clusterings with different k. If someone shows you a purity of 0.97 and no k, the first question back is how many clusters produced it.