In domain adaptation, why does re-estimating normalization statistics on target data help?

## The problem it addresses When inputs shift, the shift propagates. A channel that averaged 0.4 on source imagery might average 0.9 on target imagery; a spectral channel tuned to wideband audio sees a fraction of its usual energy on narrowband audio. Every layer downstream was tuned assuming a particular operating range, so a displaced activation distribution degrades the representation progressively rather than at one point. Networks that normalize across the batch handle this **during training** by standardising activations with statistics computed from the current batch. At inference there is no meaningful batch to compute from, so they use stored dataset-level estimates — one running mean and one running variance per channel, accumulated over source training. Those constants are the fossilised memory of the source distribution. ## The adaptation AdaBN (adaptive batch normalization) is the observation that those constants are **estimates from data, not learned parameters**, so they can be re-estimated on any data you have. The procedure: 1. Take the trained network and freeze every weight, including the per-channel scale and shift that normalization layers learn. 2. Push unlabelled target data through it in forward passes only, accumulating per-channel means and variances of the pre-normalization activations at each normalization layer. 3. Replace the stored source statistics with the target ones. 4. Serve. No labels are consulted, no loss is computed, no gradient is taken. A few hundred target batches are usually enough for stable per-channel moments; the estimate is an average, so its noise falls with the amount of data, not with any optimisation schedule. ## Why it works at all It works to the extent that the domain gap shows up as a **shift and rescaling of activation distributions** rather than a change in their shape or in which features matter. That is a surprisingly common case: changes in sensor, gain, lighting, bandwidth, compression or rendering pipeline move channel means and variances hard while leaving the underlying structure — edges, phonetic units, textures — recognisable to the same filters. Restoring each channel to zero mean and unit variance on the target puts the following layer back in the regime it was trained for, and the correction compounds favourably down the stack in the same way the error compounded. ## Why it is the right first move It costs one inference pass over unlabelled data. It cannot overfit, because it fits two numbers per channel from a large sample and touches nothing else. It is reversible: keep both sets of statistics and switch. And it gives every heavier method a fair yardstick — if adversarial alignment or self-training does not clearly beat refreshed statistics, it is not paying for its complexity, its instability and its extra training run. ## Where it stops - **Only moments.** If the shift changes which features are diagnostic, or introduces structure the source filters never learned to respond to, re-centring does nothing. Recognising a new object category is not a moment problem. - **Statistics are per domain.** If traffic mixes both domains, one blended set of statistics is wrong for both. You need to route by domain, or keep per-domain statistics, or accept a compromise. - **Batch composition matters when statistics are computed live.** Estimating from small or class-skewed batches at serving time gives noisy, input-dependent normalization and can make single-example predictions depend on their neighbours in the batch. - **Architecture dependence.** Networks that normalize each sample across its own features carry no dataset-level running statistics, so there is nothing to refresh; this baseline simply does not exist for them, and adaptation has to go through the weights. - **It is a floor, not a ceiling.** On large gaps it recovers a fraction of the drop. Treat a large residual gap as the signal to move to methods that actually change the representation. ## What to report Because the method is free, report it as a row in every adaptation comparison: source model as-is, source model with refreshed target statistics, then each heavier method. Reviewers and interviewers both look for that middle row, and its absence is usually where an over-claimed adaptation result falls apart.