What does a difference-in-differences estimate compute from a two-group, two-period panel?
answer
- two subtractions, not one
- cancels a fixed gap between groups
- cancels a shock hitting both groups
- the interaction coefficient in the regression
basics
~20 sIt subtracts the comparison group's before-to-after change from the treated group's before-to-after change. That double difference cancels the fixed level gap between the groups and any shock that moved both of them over the same window.
solid answer
~50 sDifference-in-differences uses four means: the treated group before and after, and the comparison group before and after. The estimate is `(treated_after - treated_before) - (control_after - control_before)`. The first difference removes anything about the treated group that stayed constant across the window — its baseline level, its composition, its market. The second difference removes whatever moved both groups between the two periods — seasonality, a macro shift, a platform-wide change. What is left is attributed to the treatment. The same number falls out of a regression of the outcome on a treated-group indicator, a post-period indicator, and their interaction: the interaction coefficient is the estimate, and the regression form gives you standard errors and room for covariates. The two groups do not need equal starting levels; the design assumes only that their changes would have matched.
go deeper
Be ready to compute the double difference from four means on the spot, and to say in one sentence what each of the two subtractions removes.
Expect to write the regression form — group indicator, post indicator, interaction — and explain why the interaction coefficient is the estimate and what the other two coefficients absorb.
Show you know what the number is an estimate of: the effect on treated units over the observed window, with standard errors clustered at the level treatment was assigned.
Own the framing decision: whether a two-period double difference is the right readout at all, given the rollout shape, the length of panel you have, and how reversible the decision is.
## The four numbers Difference-in-differences (DiD) is the simplest credible way to get a causal number when you could not randomise. You need two groups — one that gets the treatment and one that does not — observed in two periods, one before the treatment starts and one after. That gives you a 2x2 table of average outcomes: | | Before | After | | --- | --- | --- | | Treated group | A | B | | Comparison group | C | D | The estimate is `(B - A) - (D - C)`. Equivalently, it is `(B - D) - (A - C)`: the post-period gap between the groups minus the pre-period gap. Both orderings give the same number, and each one makes a different intuition visible. The first says *compare the changes*. The second says *compare the gaps*. ## What each subtraction buys The first difference, `B - A`, is a simple before-and-after on the treated group. It removes everything about that group that did not change over the window: how big its market was, how its users were composed, whatever fixed advantage or disadvantage it started with. What it does not remove is time. If the whole market grew 8% that quarter, `B - A` contains that 8%. The second difference, `D - C`, is the comparison group's before-and-after. Under the design's assumption, that change is an estimate of what would have happened to the treated group anyway. Subtracting it strips out the common movement. This is the step that makes DiD more than a naive before-after. So DiD needs neither group to look like the other in *level*. A treated market that is three times the size of the comparison market is fine — the size difference sits in both `A` and `B` and cancels in the first difference, and it sits in the pre-period gap and cancels in the second ordering too. What must match is the *change*, and that requirement is the parallel-trends assumption. ## The regression form In practice you rarely compute four means by hand. You fit `Y = b0 + b1*Treated + b2*Post + b3*(Treated x Post) + error` on the stacked observations, where `Treated` is 1 for units in the treated group and `Post` is 1 for observations in the after period. Then: - `b0` is the treated-group-excluded baseline: the comparison group before treatment. - `b1` is the fixed level gap between the groups. - `b2` is the common time movement affecting both groups. - `b3` is the difference-in-differences estimate — the extra movement in the treated group after treatment began. The regression form is the one to reach for because it generalises: you can add covariates, add unit and period fixed effects to move beyond two groups and two periods, and it gives you standard errors. Those standard errors should be clustered at the level treatment was assigned — usually the unit (state, market, restaurant), not the observation — because outcomes within a unit are correlated over time and ignoring that badly understates the uncertainty. ## The canonical example The study that made the design famous compared fast-food employment in New Jersey, which raised its state minimum wage in April 1992, with fast-food employment in eastern Pennsylvania, which did not. Employment levels in the two areas were not identical to begin with — that was never the point. The claim was that, absent the wage change, employment in the two neighbouring areas would have moved together. Under that claim, the double difference is the effect of the wage increase, and the finding was that New Jersey employment did not fall relative to Pennsylvania's. ## What the number means The estimate is an **average treatment effect on the treated (ATT)**: the average effect for the units that actually received the treatment, over the post-period you observed. It is not the effect the comparison group would have experienced, and it is not the effect over a longer horizon. Two honest qualifications belong in any report of it: the window it covers, and the assumption it rests on. Finally, DiD is not magic. It is a subtraction that is only as good as the claim that the comparison group's change stands in for the treated group's counterfactual change. Everything hard about the design lives in defending that claim, not in the arithmetic.
- Which coefficient in a regression with group, period and interaction terms is the difference-in-differences estimate?The coefficient on the interaction of the treated-group indicator with the post-period indicator. The group indicator absorbs the fixed level gap between the groups, the post indicator absorbs the movement common to both periods, and the interaction picks up the extra movement in the treated group after treatment started — that is the estimate.
- Does difference-in-differences estimate the effect on everyone, or only on the treated?Only on the treated — the average treatment effect on the treated, over the post-period you observed. It says nothing about how the comparison group would have responded had they been treated, and nothing about periods outside your window. Report the horizon alongside the number.
- Why is a simple before-and-after on the treated group usually not enough?Because everything else that changed between the two periods is baked into that difference: seasonality, a pricing change, a macro shift, a platform release. The comparison group's before-and-after change is your estimate of that common movement, and subtracting it out is the whole point of the design.
Two hikers set off from different altitudes. You do not compare where they stand — you compare how much each one climbed in the same hour, and the difference is what the extra gear was worth.
saying these in an interview costs you the question
- Reports the treated group's before-after change as the effect
- Claims the two groups must start at equal outcome levels
- Treats the comparison group's post-period mean as the counterfactual
- Calls the estimate causal without naming any assumption