Object Detection Heads

## What a detection head has to produce An image can contain zero objects or two hundred, but a convolutional network produces a fixed-shape tensor. Detection heads resolve this by predicting densely and filtering afterwards: the head runs over a feature map and, at every spatial location, emits a fixed number of candidate boxes with scores. Most of those candidates are background and are thrown away at inference. The design question this leaf is about is how a *location* becomes a *box*. ## Anchors as priors An anchor (also called a prior or a default box) is a rectangle of predefined size and shape, notionally centred on the feature-map cell. A typical head defines, say, three scales times three aspect ratios, giving nine anchors per location; a feature map of 50x50 then carries 22,500 anchors, and a multi-level head carries far more. For each anchor the head predicts: - a score — either one objectness value plus class scores, or a score per class directly; - four regression numbers `t_x, t_y, t_w, t_h`. The regression numbers are not pixel coordinates. The standard parameterisation is ``` b_x = a_x + a_w * t_x b_w = a_w * exp(t_w) b_y = a_y + a_h * t_y b_h = a_h * exp(t_h) ``` where `a_*` is the anchor and `b_*` the predicted box. Two properties matter. Dividing the centre shift by the anchor's own size makes the target scale-invariant: moving a small box by five pixels and a large box by fifty are the same target magnitude. Predicting a *log* size ratio makes the target symmetric around zero and guarantees a positive width and height whatever the network outputs. The result is that the network never learns to produce a box from nothing — it learns a small correction to a box that is already roughly right. ## The assignment rule Anchors also solve the labelling problem. Before you can compute a loss you must decide which of the tens of thousands of predictions is responsible for which ground-truth object. The classical rule compares each anchor to each ground-truth box by overlap: - overlap above a high threshold (commonly around 0.5 to 0.7): positive, trained to classify as that object and to regress toward its box; - overlap below a low threshold (commonly 0.3 to 0.4): negative, trained toward background; - in between: ignored, contributing no loss, so borderline cases do not push the classifier in a random direction. A rescue rule is normally added: for every ground-truth box, its single best-matching anchor is forced positive even if no anchor clears the threshold. Without it, objects whose shape is unlike any anchor would receive no supervision at all. ## The cost of anchors Anchors are hyperparameters, and they are hyperparameters coupled to the dataset. Scales and aspect ratios chosen for upright pedestrians describe tall thin boxes; the same set applied to overhead imagery, where an object's axis-aligned box changes aspect ratio continuously as the object rotates, may leave whole shape ranges without a well-matching anchor. Those objects get few positives, are under-trained and are missed at inference. Practitioners respond by clustering box shapes in the training set to pick anchor sizes, by tuning the thresholds, or by adding more anchors — which raises memory and the already extreme background-to-foreground ratio without necessarily helping. ## Anchor-free heads Anchor-free heads keep the dense grid and delete the reference boxes. The two common families are: **Centre-based.** A location is positive if it falls inside a ground-truth box (often restricted to a small region around the box centre). It predicts four non-negative distances — to the left, top, right and bottom edges — so the box is recovered as `(x - l, y - t, x + r, y + b)` from the location's own coordinates. Because locations near the border of a large object produce poor boxes, such heads usually add a per-location quality score, sometimes called centre-ness, that downweights off-centre predictions when boxes are ranked. **Keypoint-based.** The head predicts heatmaps of box corners or centres and then pairs or sizes them, treating detection as keypoint localisation rather than box regression. The honest summary is that anchor-free is not assignment-free. You still choose which locations are positive, how objects of different sizes map onto pyramid levels, and how ambiguous overlapping objects are resolved. What you gain is fewer geometric hyperparameters, fewer predictions per location, and no dependence on the box-shape statistics of the training set; what you give up is the built-in prior that made the regression target small and easy, which is why anchor-free heads care more about their quality-score and assignment design. ## What an interviewer is listening for That anchors are constants, not learned weights. That the head predicts a *correction*, not coordinates. That the overlap thresholds are a labelling device, distinct from anything done at inference. And that switching to anchor-free moves the difficulty rather than removing it.