Why does a per-RoI mask branch predict one binary mask per class instead of a per-pixel softmax over classes?

## Where the branch sits Instance masks are usually produced by adding a third head to a two-stage detector. The first stage proposes regions; for each region the head already runs a classification branch (which class, or background) and a box-regression branch (refine the rectangle). The mask branch is a small stack of convolutions on the same pooled region features, ending in a set of low-resolution maps — a common choice is 28x28 — that are then resized onto the predicted box to give a full-resolution mask. The design question is what those maps should represent. ## Two candidate parameterisations **Per-pixel multinomial.** Emit one map with `K` channels and take a softmax across channels at each pixel, exactly as semantic segmentation does. Each pixel is assigned to one class or background, and the classes compete: raising `dog` at a pixel necessarily lowers `cat` there. **Per-class binary.** Emit `K` independent maps, each with a per-pixel sigmoid. Map `k` answers, for every pixel in the region, "is this pixel part of the object, assuming the object is of class `k`". No cross-channel normalisation, so the channels do not compete. Mask heads use the second, and the training rule is the other half of the design: the loss is the average binary cross-entropy over the pixels of the ground-truth class's channel *only*. The remaining `K-1` channels receive no gradient from that region. ## Why decoupling helps **The class question is already answered elsewhere.** Making the mask branch also discriminate between classes duplicates work the classification branch does with better features and a cleaner objective — it sees the whole region, not one pixel at a time. **Competition couples two error modes.** Under a softmax, a region whose class is genuinely ambiguous spreads probability across several channels, so no channel confidently claims the object's pixels and the *shape* degrades even when the pixel set is obvious. Under independent sigmoids, an ambiguous class costs you the label, not the silhouette; the mask stays a clean foreground/background call. **The per-pixel task becomes trivial to state.** "Foreground or not, within this box" is a binary problem with strong local evidence and roughly balanced positives and negatives inside a tight region. That is a far easier learning problem than a `K`-way decision at every pixel with almost all pixels belonging to a handful of classes. **Class-specific shape priors survive.** Because each channel is trained only on regions of its own class, channel `k` can specialise in what a `k` looks like — the silhouette of a person is not the silhouette of a car — without the specialisation being expressed as a competition. ## Practical consequences *Inference reads one channel.* The classification branch's chosen class selects the channel; the score attached to the instance is the classification score, not a mask score. A consequence is that a badly classified object can still produce a good-looking mask that ends up filed under the wrong class. *Only one channel is supervised per region.* People are often surprised that the other channels are not pushed towards zero. They are simply not part of that region's loss, which keeps the objective from spending capacity on suppressing masks nobody will read. *The mask lives in the region's own frame.* It is predicted at fixed low resolution relative to the box, then resized. That makes the branch cheap and box-size-independent, but it caps boundary detail: thin structures and sharp corners are smoothed by the resize, and a mask can be no better than the box it is painted into. Raising the mask resolution recovers some detail at extra compute. *Overlap is allowed.* Because masks are predicted per region independently, two instances can claim the same pixel. That is fine for instance segmentation and is precisely the loose end that has to be tied off when the output is merged into a panoptic labelling. ## What a weak answer looks like Saying "binary is simpler" without naming the coupling that softmax introduces, or claiming that all `K` channels are trained on every region, or asserting that the mask branch classifies the object. The point to land is that the head has *already* decided the class, so making the mask decision independent of it is free accuracy.