A ten-layer stack of stride-1 3x3 convolutions misses a 300-pixel lesion; what is its receptive field and how do you grow it?

## Definition The **receptive field** of a unit in a convolutional network is the set of input pixels that can affect that unit's value. It is a purely structural property: it depends on kernel sizes, strides and the layer ordering, and not at all on the learned weights. If a pixel lies outside a unit's receptive field, no training procedure and no amount of data can make that unit sensitive to it. Two quantities are tracked together as you walk the stack forward: - `r` — the current receptive field size, in input pixels. Starts at 1. - `jump` — how many input pixels separate two adjacent positions of the current feature map. Starts at 1. For each layer with kernel size `k` and stride `s`: ``` r <- r + (k - 1) * jump jump <- jump * s ``` Read the first line carefully: a layer widens the field by `(k - 1)` **steps of the current map**, and each of those steps is worth `jump` input pixels. This is why stride matters so much: it does not widen the field on the layer where it appears so much as it multiplies the value of every widening that comes after. ## The stated stack Ten stride-1 3x3 convolutions. `jump` stays 1 throughout, and each layer contributes `(3 - 1) * 1 = 2`. So ``` r = 1 + 10 * 2 = 21 ``` Twenty-one pixels. A unit at the top of this stack sees a 21x21 patch of the image. Asking it to decide the presence of a 300-pixel lesion is asking it to judge a structure from about one twentieth of the structure's area. The model will do what a model always does when the evidence is missing: latch onto whatever local texture happens to correlate with the label in the training set, then fail out of distribution. Depth alone gave the network parameters, not perception. A useful sanity check when a network underperforms on large objects: compute this number before touching the learning rate. ## Lever 1 — more depth Each additional stride-1 3x3 layer adds 2 pixels, so the field grows **linearly** in depth. Reaching 300 pixels needs roughly (300 - 1)/2 ≈ 150 such layers. That is an enormous amount of computation, memory and optimisation difficulty bought purely to widen a window. Depth is the least efficient lever there is for this problem, which is the point most candidates miss when they propose "make it deeper" as the whole answer. ## Lever 2 — downsample A layer with stride 2 doubles `jump`. Every subsequent layer's `(k - 1)` steps are then worth twice as many input pixels, so the growth becomes **multiplicative** in the number of downsampling stages. Take the same ten 3x3 convolutions and insert a 2x2 stride-2 pool after every second one — four reductions in total: ``` conv, conv -> r = 5, jump = 1 pool -> r = 6, jump = 2 conv, conv -> r = 14, jump = 2 pool -> r = 16, jump = 4 conv, conv -> r = 32, jump = 4 pool -> r = 36, jump = 8 conv, conv -> r = 68, jump = 8 pool -> r = 76, jump = 16 conv, conv -> r = 140, jump = 16 ``` The same ten convolutions now reach 140 pixels instead of 21, and they do so on grids that are progressively cheaper to compute. This is the main reason downsampling stages exist at all in a classification trunk; the compute saving is a bonus, not the motivation. The cost is resolution: the final grid is coarse, and `jump = 16` means neighbouring output positions are 16 input pixels apart, so anything that must be localised precisely has lost that precision. For a pure classification decision this is fine. For a task that must draw a boundary, downsampling to get context and then restoring resolution is the design tension the architecture has to resolve. ## Lever 3 — larger kernels A `k x k` layer adds `(k - 1) * jump` rather than `2 * jump`. A 7x7 kernel adds three times as much per layer as a 3x3. The cost is quadratic in `k` for parameters and multiply-adds at that layer, which is why deep stacks of small kernels historically displaced shallow stacks of large ones — but as a deliberate early-layer choice, a larger kernel is a cheap way to start the field off wider before the downsampling stages begin. ## The theoretical field is an upper bound The number the recurrence gives you counts every pixel that *can* influence the output. It does not say every pixel influences it equally. Contributions accumulate through many overlapping paths, and there are far more paths reaching the centre of the field than the corners; the resulting weighting is approximately Gaussian, densest at the centre and decaying outward. The **effective** receptive field — the region actually carrying meaningful influence — is therefore a fraction of the nominal one, and that fraction shrinks as the stack deepens, because the nominal field grows faster than the effective one does. The corner pixels of a nominal 140-pixel field may contribute almost nothing. The practical consequence: treat the computed number as an optimistic ceiling. If your object is 300 pixels wide, a computed field of exactly 300 is not comfortable — you want headroom, typically a nominal field several times the object's extent. ## How to check empirically Two cheap diagnostics. Occlude a patch of the input and see whether a chosen output unit moves at all — outside the field it cannot. Or take the gradient of a single centre unit with respect to the input pixels and look at the magnitude map: it is nonzero only inside the theoretical field, and its shape reveals the centre-weighted decay of the effective field directly.