Your detector finds buses reliably but misses 16-pixel traffic signs. Why does a single-scale head fail here?

## Where small objects disappear A convolutional backbone reduces spatial resolution in stages, typically to strides of 4, 8, 16 and 32 relative to the input. Stride is the key number: at stride 32, one feature-map cell corresponds to a 32x32 pixel patch of the original image. Now put a 16-pixel traffic sign into that. It fits entirely inside one cell, and it does not even fill it — most of the cell's evidence is background. Everything that distinguished the sign, the few pixels of shape and colour, has been averaged and strided away through four downsampling stages. A bus 300 pixels wide covers roughly nine by nine cells at the same stride and loses nothing that matters. A second, quieter failure runs alongside the first. Assignment is geometric. On an anchor-based head, the smallest anchor at stride 32 may be, say, 32 pixels on a side; a 16-pixel ground-truth box overlaps it at well under the positive threshold, so no anchor is labelled positive for it and the object contributes almost no learning signal beyond a best-match rescue. On an anchor-free head, the object may cover so few locations that it is represented by a single noisy one. Either way the model is not merely bad at small objects — it was barely trained on them. ## Why not just use the shallow layers The obvious response is to attach the head to a stride-4 feature map, where the sign covers sixteen cells. The problem is that early features are generic: edges, colours, simple textures. They have the resolution to localise but not the semantics to classify. A head there will fire on every high-contrast blob. The deep features have the opposite profile — rich semantics, no resolution. A pyramid is the construction that refuses to choose. ## The feature pyramid A feature pyramid builds one prediction level per backbone stage: 1. Take the deepest stage as the top pyramid level. 2. Upsample it by two and add a *lateral connection*: the corresponding backbone stage, passed through a small projection so the channel counts match. 3. Repeat down the pyramid. Each resulting level therefore carries semantics that flowed down from the deep stages and spatial detail that came in sideways from the shallow stage at that resolution. The detection head is then applied to every level, usually with shared weights, which keeps the parameter cost close to that of a single-scale head. Assignment becomes size-based: an object is routed to the level whose stride suits it, commonly by a rule on the square root of its box area, so small objects train and predict on the fine levels and large objects on the coarse ones. Two consequences follow. Each level sees a narrow range of object sizes, which makes the regression easier. And a head with shared weights across levels sees a roughly scale-normalised problem, which is part of why it generalises across sizes. ## The alternatives and what they cost **Raise the input resolution.** Feeding a larger image gives the sign more pixels and pushes it up into a size range the strides already handle. It is effective and it is expensive: compute and memory grow roughly with the pixel count, and everything else in the frame gets slower too. **Keep resolution in the backbone.** Removing a stride in the last stage and using dilated convolutions there preserves the feature-map resolution while keeping the receptive field large, since dilation spreads the kernel's sampling points apart without adding parameters. The cost is that the deep stage now runs at four times the spatial size, which is a substantial compute and memory increase. **Tile the image.** Splitting a large frame into overlapping crops and running the detector on each raises the effective resolution per object without changing the model, at the cost of several forward passes per frame and a stitching step that must reconcile detections across crop boundaries. **Fix the anchors or the assignment.** If the strides are fine but the smallest anchor is too large, adding smaller anchors or relaxing the positive threshold for tiny objects can recover a surprising amount, and it is nearly free. Always check this before rebuilding the architecture. ## Diagnosing it properly The symptom to establish first is that the failure is size-dependent, not class-dependent. Bin the validation set by ground-truth box area and look at recall per bin: a clean monotonic collapse below some pixel size points at resolution and assignment; a failure spread evenly across sizes for one class points at something else, such as appearance or label noise. Then check whether the small objects even receive positive assignments during training — if they do not, no architectural change downstream will help until assignment is fixed. ## What an interviewer is listening for The word *stride*, and the arithmetic that turns it into a cells-per-object number. The distinction between losing the signal and never assigning the object. The reason shallow features alone are not the answer, which is where the top-down path earns its place. And a ranking of fixes by cost, with the cheap assignment check before the expensive resolution change.