What does the Mann-Whitney U test actually compare between two independent samples?

## The statistic Take two independent samples of sizes n1 and n2. Pool all N = n1 + n2 observations, sort them, and assign ranks 1 to N, giving tied values their midrank. Let R1 be the sum of the ranks that landed in group 1. Then ``` U1 = R1 - n1(n1 + 1)/2 U2 = n1*n2 - U1 ``` U1 has a direct combinatorial meaning: it is the number of (x, y) pairs, one value from each sample, in which the group-1 value is the larger, with each tie counted as one half. There are n1*n2 such pairs in total, which is why U1 + U2 = n1*n2. Under the null hypothesis the sampling distribution of U is known exactly for small samples and is approximately normal for larger ones, with ``` mean(U) = n1*n2/2 var(U) = n1*n2*(N + 1)/12 (before any tie correction) ``` Heavy ties shrink the true variance, so implementations apply a tie correction that reduces the variance term; skipping it makes the test conservative. This test is exactly the Wilcoxon rank-sum test under a different but equivalent parameterisation — interviewers use the two names interchangeably. ## What the null hypothesis actually says The textbook null is that the two samples are drawn from the **same distribution**. Its most useful restatement is ``` P(X > Y) + 0.5 * P(X = Y) = 1/2 ``` in words: pick one observation from each group at random and either is equally likely to be the bigger one. The estimate of that quantity is `U1/(n1*n2)`, sometimes called the probability of superiority or common-language effect size. This is why U is often described as a test of **stochastic ordering** rather than of any single summary number. The alternative hypothesis in the general form is simply that the two distributions differ in a way that makes one tend to produce larger values. ## Why it is not automatically a median test This is where candidates most often slip. If the two distributions have the same shape and spread and differ only by a horizontal shift, then stochastic ordering and a median difference are the same statement, and reading a significant U as "group A tends to score higher, and its median is higher" is fair. Add a matching point estimate of the shift — the median of all pairwise differences x - y, the Hodges-Lehmann estimator — and the result is interpretable. Drop the equal-shape assumption and the equivalence breaks. Two groups can have identical medians while one is far more dispersed than the other; U can then reject, correctly, because the distributions genuinely differ, yet a claim about medians would be wrong. When shapes plainly differ, the Brunner-Munzel test targets the same `P(X < Y)` quantity without assuming a common shape and is the more honest choice. ## Where it fits naturally Ordinal outcomes are the clearest case. Satisfaction ratings on a 1-to-5 scale carry order but not distance: nothing guarantees that the step from 1 to 2 means as much as the step from 4 to 5, so an average of stars is arithmetic on labels. Ranks use only the ordering the scale really supports. Expect a great many ties on a five-point scale, which makes the tie correction and the normal approximation, rather than the exact null distribution, the relevant machinery. Strongly skewed continuous measurements are the other common case: session duration, time-to-repair, spend per user. Here you should be explicit that a rank comparison answers *how often* one group is higher, not *by how much* — and if the decision is denominated in the total or the average, that difference matters. ## Assumptions worth stating out loud 1. Observations are independent within each group and the two groups are independent of each other. Repeated measurements on the same subject break this; that design calls for a paired rank method instead. 2. The outcome is at least ordinal. 3. For a location interpretation only: similar shapes and spreads. Notably absent: normality, and any requirement that variance be equal — as long as you keep the interpretation to stochastic ordering rather than to medians. ## Reporting Quote U (or the rank sum), the sample sizes, the p-value, and an effect size. The two standard rank effect sizes are the probability of superiority `U1/(n1*n2)` and the rank-biserial correlation `2*U1/(n1*n2) - 1`, which runs from -1 to +1 and is zero when the groups are indistinguishable. A p-value with no effect size is an incomplete answer at any level above screening.