Assumption Checks

## What the test actually needs A two-sample t-test asks whether two population means differ. It builds a statistic of the form `t = (mean1 - mean2) / SE`, where `SE` is the estimated standard error of the difference, and then compares that statistic to a t distribution. Everything the test assumes exists to make that last step true: that the numerator is approximately normal around the true difference, and that the denominator is an independent estimate of its spread with the stated degrees of freedom. ## Assumption 1 — independence Observations must be independent both within each group and between the groups. This is a property of how the data were produced, and it cannot be checked by looking at the numbers alone. You establish it by asking what a single row represents and whether any unit contributes more than one row: repeated measurements on the same subject, multiple sessions from one user, pupils inside a classroom, orders inside a household, consecutive readings from one sensor. When rows are correlated, the standard error is computed as if there were far more information than there really is, so it comes out too small; intervals are too narrow and p-values too optimistic. Crucially, collecting more correlated rows makes this worse, not better. Independence is also why random sampling (or random assignment) matters: it is the design feature that buys the assumption. ## Assumption 2 — normality, of the right thing The common misstatement is that the raw data must be normal. What the test needs is that the sampling distribution of each group mean is approximately normal. Normal raw data is a sufficient condition for that, not a necessary one. The central limit theorem says that as sample size grows, the distribution of a sample mean approaches normality for any population with finite variance. How fast depends on the shape: mildly skewed data converge quickly, and by a few dozen observations per group the approximation is usually good; strongly skewed or heavy-tailed data need far more. Two situations therefore deserve care. Small samples, where the theorem has not had a chance to work, so raw-data shape matters directly. And heavy tails or outliers at any size, because a single extreme value inflates the sample standard deviation, which is what the denominator of the statistic is built from — the error rate may be fine while power quietly collapses. You check this with a normal Q-Q plot of each group: sorted values against the values a normal distribution would place at those positions. A systematic bend tells you the direction and size of the departure. A formal normality test tells you far less, because its verdict tracks sample size more than it tracks any departure that matters. ## Assumption 3 — equal variances The classical pooled version combines both groups' variances into one estimate, which is only sensible if the populations share a variance. Compare the two sample standard deviations, or use a spread test — Levene's, which works on absolute deviations from a group centre, is the robust choice. Unequal variance matters most when the group sizes are also unequal: with balanced groups the pooled procedure tolerates a fairly large variance ratio, whereas an imbalanced design where the smaller group has the larger variance pushes the true error rate above nominal. A procedure that does not pool the variances removes this assumption entirely, which is why many practitioners default to one. ## A fourth, quieter requirement The mean has to be a meaningful summary. On a strongly skewed outcome the mean is not what a stakeholder pictures, and on an ordinal scale it may not be defined at all. Passing every assumption check does not make the mean the right question. ## How to talk about it in an interview Name the three, then rank them. Say that normality applies to the sampling distribution of the mean and that the central limit theorem covers it at reasonable sample sizes; say that equal variance is only a requirement of the pooled version; say that independence is structural, undetectable from the numbers alone, and the one that more data makes worse. Add that assumption checks should be decided before you see the outcome — choosing the test after seeing which one gives the smaller p-value invalidates the error rate you are reporting.