In a CNN, what is the difference between translation equivariance and translation invariance?

## Two properties, stated precisely Let T be an operation that shifts a spatial input by some number of pixels. For a function f — a layer, a stack of layers, or a whole network: - f is **equivariant** to that shift if `f(T x) = T f(x)`. Shift the input, and the output changes in the same, predictable way: it shifts too. Information about *where* is preserved. - f is **invariant** to that shift if `f(T x) = f(x)`. Shift the input, and the output is identical. Information about *where* has been discarded. These are not degrees of the same thing. Equivariance says the output tracks the change; invariance says the output ignores it. A network can be equivariant in its body and invariant at its output, and that is the usual design. ## Why weight sharing produces equivariance A convolution applies the same kernel window at every position. If a pattern that produced a strong response at position p is moved to position p + d, the identical window arrives over it at p + d and produces the identical response there. Nothing about the layer's parameters is position-specific, so responses move exactly as the input does. That is the direct consequence of sharing, and it is what people mean when they say convolution has a built-in translation prior. Notice this gives no invariance whatsoever. A feature map that shifts is a *different* feature map. If you flattened it and fed it into a position-sensitive dense readout, the network's output would change under a shift. ## Where invariance actually comes from Invariance has to be introduced deliberately, and there are essentially three routes: 1. **A readout that discards position.** If the final step aggregates each feature map over all spatial locations into a single number per channel, then a shifted map produces the same aggregate, so the classifier output is unchanged. This is the standard route for a whole-image classification head. 2. **Downsampling that coarsens position.** Reducing spatial resolution makes small shifts stop mattering at the coarse scale, which yields partial, local invariance rather than exact invariance. 3. **Learning it from the data.** If the training set contains the object at many positions with the same label, the network can learn a response that happens to be roughly position-independent, even where the architecture does not enforce it. ## The conveyor-belt case A fixed camera photographs parts on a belt and a model must output pass or fail. A scratch appears 40 pixels further along in one frame than in another. Through the convolutional body, the activation caused by that scratch moves along with it — that is equivariance doing its job, and it is exactly what you want, because the same detector must work at both positions without extra parameters. At the output, though, the two frames must agree: the part is defective either way. That agreement is invariance, and it is supplied by the aggregating readout, not by the convolutions. If you replaced that readout with something position-sensitive, you would get two different scores from two frames that show the same defect, and no amount of extra convolution depth would fix it. ## Equivariance is only approximate in real networks Three things break the clean equality: - **Borders.** Padding at the edges means a pattern near the boundary is not treated identically to the same pattern in the middle, and content shifted off the edge is simply gone. - **Stride and downsampling.** A layer with stride s is only cleanly equivariant to shifts that are multiples of s. Compose several such layers and only shifts that are multiples of the cumulative stride map exactly onto grid positions. - **Aliasing.** Because downsampling samples the response on a coarse grid, a shift of a single pixel can change which peak lands on a sample point, which is why predictions can wobble under a one-pixel shift even though the architecture is nominally shift-friendly. ## When you want to keep equivariance rather than collapse it Dense prediction tasks depend on equivariance surviving all the way to the output. In segmentation the mask must move when the object moves; in detection the box must move; in keypoint or depth prediction the output map is spatial by definition. For those, you deliberately avoid an aggregating readout that would destroy position, and the equivariance of the convolutional body is the property that makes a spatially structured output learnable at all. ## And it is translation only Weight sharing gives a prior about *translation* and nothing else. A convolutional stack is not automatically equivariant or invariant to rotation, scale, reflection, or lighting change. Those need either a different architectural construction or training data that covers the variation. Claiming that a convolution handles rotation for free is a frequent and easily caught mistake. ## Interview framing The crisp answer is: convolution gives equivariance, the readout gives invariance, and knowing which one your task needs at the output is the actual design decision.