When is ten-crop plus flip test-time augmentation worth its ten-fold inference cost?

## What test-time augmentation is Everywhere else on this topic, augmentation is a training-only device and evaluation runs on a fixed deterministic pipeline. Test-time augmentation is the deliberate exception. At inference you build several views of the same input, run each through the unchanged model, and combine the outputs into one prediction. Ten-crop is the classic recipe for image classification: take four corner crops and one centre crop of the resized image, then add the horizontal mirror of each. Five crops times two orientations is ten views. Each view is passed through the network and the ten output probability vectors are averaged into the final prediction. Nothing about the weights changes. TTA is a decision-rule change, not a training change. ## Why it helps A single centre crop is one sample of a somewhat arbitrary choice - which part of the frame the model sees and in which orientation. The prediction inherits variance from that choice. Averaging over a set of views is variance reduction on the prediction, the same statistical idea that makes an ensemble of models work, applied to an ensemble of inputs instead. It is cheapest exactly where the model is uncertain: an object that falls near a crop boundary is captured properly by at least some of the views. The gain is usually small. It is also usually real, and it costs no retraining, which is why it appears in evaluation protocols so often. ## Aggregation choices Average the predicted probabilities. Two alternatives are worse or different: - **Hard voting** over the ten argmaxes throws away the margin. Nine views at 0.51 and one at 0.99 for the other class is information you should keep, and votes can tie. - **Averaging logits** before the softmax is a defensible but distinct operation - roughly a geometric rather than arithmetic mean of probabilities. It weights confident views more heavily and shifts calibration. Pick one deliberately and keep it consistent between evaluation and serving. Whatever you choose, the aggregation is now part of the model's contract. Calibration measured on single-crop outputs does not describe the averaged output. ## The cost, stated honestly Ten views is ten forward passes. If they run serially, request latency multiplies by roughly ten and your tail latency is where the pain lands. If you batch the ten views into one forward pass you convert most of the latency hit into a throughput hit, but the compute bill is unchanged: your serving fleet does ten times the work per request, so cost per prediction is ten times higher. That framing is the interview answer. The question is not *does TTA help* - it usually helps a little - but *is this the best use of a ten-fold compute budget?* Often it is not. The same compute spent on a larger model, a higher input resolution, or a genuine ensemble of two or three differently-seeded models typically buys more accuracy than ten crops of one model. ## When it is the right call - **Offline or batch scoring**, where throughput matters and per-item latency does not. - **Evaluation and reporting**, provided you say clearly that the number is a TTA number, since it is not comparable to single-pass results. - **High-stakes, low-volume inference**, where a small accuracy gain is worth real money and traffic is thin. - **Cascades**: run a single pass, and fall back to TTA only for inputs whose confidence lands in an ambiguous band. Most traffic pays one pass; the hard tail pays ten. And when it is not: any tight online latency budget, any high-volume path where compute cost dominates, and any case where a cheaper single-pass change - better training, a longer schedule, a bigger backbone - is still on the table. ## Two discipline points people get wrong **The views must be label-preserving, and should match the trained invariances.** Corner crops assume the object survives cropping; if your objects routinely span the whole frame, corner crops feed the model partial evidence and TTA can hurt. Applying a flip at test time when the model was trained without flips - or, worse, in a domain where mirroring changes the answer - degrades the average. The training policy and the test-time view set should be consistent. **Select with TTA on if you serve with TTA on.** Checkpoint selection, decision thresholds and calibration all depend on the exact inference procedure. Tuning a threshold on single-crop validation outputs and then serving the ten-view average silently shifts the operating point. Whichever procedure ships must be the procedure the validation numbers came from. ## Cheaper relatives If the ten-fold bill is refused but you still want some of the effect: use two views - centre crop and its mirror - for most of the gain at a fifth of the cost; batch views together; or distil the TTA-averaged outputs into a single-pass student so the averaging cost is paid once at training time rather than on every request.