In a two-proportion z-test comparing 42/300 with 61/300, when is the standard error pooled?

## The setup Two survey arms each contacted 300 people. Arm A returned 42 responses, arm B returned 61. ``` p1 = 42/300 = 0.140 p2 = 61/300 = 0.2033 difference = 0.0633 ``` The question is whether that gap is bigger than sampling noise, and the two-proportion z-test answers it with the usual skeleton: difference divided by its standard error, compared against the standard normal. ## Why the test pools A hypothesis test computes "how surprising is this data **if the null were true**". The null here is `p1 = p2`: one shared response rate produced both arms. Under that assumption, splitting the data into two separate variance estimates throws information away — every one of the 600 contacts is a draw from the *same* Bernoulli process. So you estimate the shared rate from everything: ``` p_pool = (42 + 61) / (300 + 300) = 103/600 = 0.17167 SE_0 = sqrt( p_pool * (1 - p_pool) * (1/n1 + 1/n2) ) = sqrt( 0.17167 * 0.82833 * (1/300 + 1/300) ) = sqrt( 0.142193 * 0.0066667 ) = 0.03079 z = 0.06333 / 0.03079 = 2.06 ``` The two-sided p-value is about 0.04, so at a 5% level this is a significant difference — though only just, which is worth saying out loud rather than reporting "significant" flatly. Pooling matters because it makes the test's null distribution exactly the one the p-value claims. If you compute the statistic with an unpooled standard error and then compare it to the standard normal, the reference distribution no longer matches how the statistic behaves under the null, and the error rate drifts from its nominal level — usually slightly, but the pooled version is the one with the clean justification. ## Why the interval does not pool A confidence interval is not computed under the null. Its whole purpose is to display the range of differences consistent with the data, including differences far from zero. Estimating the standard error under an assumption that the difference is zero would be circular: you would be using "they are equal" to describe how uncertain you are about how unequal they are. So each arm contributes its own variance: ``` SE_unpooled = sqrt( p1*(1-p1)/n1 + p2*(1-p2)/n2 ) = sqrt( 0.140*0.860/300 + 0.2033*0.7967/300 ) = sqrt( 0.00040133 + 0.00053996 ) = 0.03068 CI(95%) = 0.0633 +/- 1.96 * 0.03068 = (0.0032, 0.1235) ``` The interval excludes zero, consistent with the test rejecting — as you would expect when both are computed correctly on the same data. ## When do the two standard errors actually differ? In this example, hardly at all: 0.0308 versus 0.0307. That is typical of a **balanced** design with proportions that are not far apart. The pooled and unpooled standard errors separate when: - **Group sizes are unequal.** Pooling weights the combined rate by sample size, so a large arm drags the assumed common rate toward its own value. - **The proportions are far apart.** Pooling assumes one rate; the further the truth is from that, the worse the assumption describes either arm. - **Rates are near 0 or 1**, where `p * (1 - p)` changes fastest with `p`. When they do differ, you can get the awkward boundary case where a 95% interval excludes zero while the pooled test returns `p > 0.05`, or the reverse. That is not a contradiction to be argued away — it is the honest consequence of the two procedures answering different questions. Report both, say which one your decision rests on, and do not go shopping for whichever crosses the threshold. ## Conditions for the normal approximation The z-test approximates a discrete binomial by a continuous normal, which needs enough events in each cell. The usual rough guideline is at least about 10 successes and 10 failures per group. Here 42 and 258, and 61 and 239, are comfortable. With single-digit counts in any cell the approximation degrades and an exact method is more appropriate. ## Practical checklist 1. Confirm each arm is a count of independent successes out of a fixed number of trials. 2. Check the counts are large enough for a normal approximation in every cell. 3. For the **test**: pool, `z = (p1 - p2) / sqrt(p_pool*(1-p_pool)*(1/n1 + 1/n2))`. 4. For the **interval**: do not pool, `(p1 - p2) +/- 1.96 * sqrt(p1*(1-p1)/n1 + p2*(1-p2)/n2)`. 5. Report the difference and its interval, not only the p-value — a 6.3 percentage-point gap with an interval running from 0.3 to 12.4 points is a much more useful sentence than "p = 0.04". ## What interviewers are checking That you understand a null hypothesis is not just a threshold to compare against but an assumption you are allowed to *use* while computing the test statistic — and that the same assumption becomes illegitimate the moment you switch from testing to estimating. Candidates who have only ever called a routine rarely notice that these two standard errors are different quantities at all.