What does a significant Kruskal-Wallis test tell you about three or more groups?

## The setup You have k independent groups — say three teaching methods, each tried on a different set of students — and an outcome that is ordinal or badly skewed. You want to know whether the groups differ at all before you look at any particular pair. ## The statistic Pool all N observations across the k groups, rank them from 1 to N with midranks for ties, and let `Rbar_i` be the mean rank of group i, with n_i observations in it. The overall mean rank is `(N + 1)/2`. Then ``` H = [12 / (N(N + 1))] * sum over i of n_i * (Rbar_i - (N + 1)/2)^2 ``` which is algebraically the same as `H = [12/(N(N+1))] * sum(R_i^2 / n_i) - 3(N + 1)`, using rank sums `R_i`. Read the first form: it is a weighted measure of how far the group mean ranks scatter around the grand mean rank. All groups behaving alike puts the mean ranks near `(N+1)/2` and H near zero; one group monopolising the high ranks pushes H up. When ties are present, H is divided by `1 - (sum of (t^3 - t)) / (N^3 - N)`, where t runs over the sizes of the tied groups. This inflates H, compensating for the reduced variability ties introduce. ## The reference distribution For reasonably sized samples H is compared with a chi-square distribution on `k - 1` degrees of freedom — one fewer than the number of groups, exactly as in the numerator degrees of freedom of a one-way analysis of variance. The approximation degrades with very small groups, where exact or permutation-based null distributions are preferable. A useful sanity anchor: with k = 2 the Kruskal-Wallis test reduces to the two-sided Mann-Whitney U test, and the two give the same p-value. ## What significance does and does not say A significant H is an **omnibus** result. It says: the k samples do not all come from the same distribution; at least one group tends to produce larger values than at least one other. It does not say which pair, and it does not by itself rank the groups. To locate the difference you run pairwise rank comparisons afterwards, and because the number of pairs grows as k(k-1)/2, you must control the error rate across them — Dunn's test, which compares mean ranks using the pooled ranking with an adjustment for multiplicity, is the standard rank-based follow-up. Reporting the omnibus p-value plus one eye-catching pair with no correction is a classic weak answer. The interpretive caveat carried over from the two-group case applies here too. The null is that the k distributions are identical. Reading a rejection as "the medians differ" requires that the groups share a shape and spread; if one group is much more dispersed than the others, H can be significant even with equal medians, and the honest statement is that the distributions differ. ## Assumptions 1. Observations are independent within and across groups. Students taught in the same classroom are not independent, and clustered data needs a method that models the clustering rather than a plain rank test. 2. The outcome is at least ordinal. 3. Similar shapes across groups, only if you intend a location interpretation. Normality of the outcome is not required, and neither is equal variance for the general distributional null — but equal spread does matter once you want to talk about location. ## Effect size and reporting Report H, the degrees of freedom `k - 1`, the group sizes and mean ranks, and the p-value. `epsilon-squared`, computed as `H/((N^2 - 1)/(N + 1))`, gives the proportion of rank variability attributable to group membership and runs from 0 to 1; eta-squared style variants exist as well. Then report the post-hoc comparisons with the correction you used, and the group medians so a reader can see the direction. ## A common trap Candidates sometimes say the test "compares the medians of the groups". Kruskal-Wallis never computes a median. It compares mean *ranks*. The distinction matters both for the equal-shape caveat and for interpreting mean ranks — a mean rank is a position within the pooled sample and has no meaning outside the specific data set that produced it.