What does the margin in a triplet loss enforce, and when is a triplet's loss exactly zero?

## The three roles A triplet is `(a, p, n)`: an **anchor**, a **positive** that shares the anchor's identity, and a **negative** from a different identity. The loss operates on the two distances the anchor forms: ``` L(a, p, n) = max(0, d(a, p) - d(a, n) + m) ``` where `d` is a distance between embeddings (the original FaceNet formulation uses *squared* Euclidean distance; plain Euclidean is also common, and the units of `m` follow whichever you pick — a margin is not portable between the two). ## What the margin buys Read the constraint the loss is trying to satisfy: ``` d(a, n) >= d(a, p) + m ``` Without `m`, the requirement is only `d(a,n) > d(a,p)` — an *ordering*. Orderings are cheap and fragile. They can be satisfied by an embedding where every point sits in a tiny ball and the differences are numerical dust, and they leave no room for the test-time variation (a new pose, a noisier microphone) that will jitter distances. The margin turns an ordering constraint into a *separation* constraint: there must be a band of width `m` between the positive distance and the negative distance. That band is what a single global decision threshold later lives inside. Note what the margin does *not* do. Triplet loss is **relative**: it never says how big `d(a,p)` should be in absolute terms, only how much smaller than `d(a,n)`. Different identities are free to have different intra-class spreads. That flexibility is the main reason triplet loss is often preferred to a pairwise contrastive loss, which pulls every positive pair toward zero distance unconditionally and pushes negative pairs only until they exceed the margin — a stricter, absolute geometry that can fight a class with genuinely high internal variation. ## When the loss is exactly zero Because of the `max(0, .)` hinge, the loss is exactly zero — and so is its gradient — whenever ``` d(a, n) - d(a, p) >= m ``` Call these **easy triplets**. They are already correct with room to spare, so they contribute nothing. The other regimes are worth naming: a triplet where the negative is *farther than the positive but by less than* `m` is still ordered correctly yet violates the margin (positive loss, moderate gradient), and one where the negative is *closer than the positive* is ordered wrongly (large loss). The practical consequence is a moving target: early in training most triplets are non-zero, and after a few epochs a randomly sampled batch is almost entirely easy triplets, so the average gradient collapses towards zero even though nothing is broken. The number of *active* triplets in a batch, not the loss value alone, is the quantity to watch. ## Embedding normalisation and the units of the margin There is a trivial way to satisfy the margin that has nothing to do with identity: scale every embedding up. Multiplying the whole space by 10 multiplies every distance by 10, so any fixed `m` becomes easy to clear without the geometry improving at all. The standard defence is to **L2-normalise** the embeddings, projecting them onto the unit hypersphere before the distances are computed. Then vector norms are pinned at 1, the Euclidean distance between any two embeddings is bounded in `[0, 2]` (so squared distance is bounded in `[0, 4]`), and the margin is a scale-free quantity meaning the same thing everywhere in the space. That also makes a single global verification threshold coherent, and it is why margins on normalised embeddings are small numbers — a value such as 0.2 with squared distances is a typical published choice. ## Choosing the margin The margin is a real hyperparameter with a two-sided failure mode. Too small and almost every triplet clears it immediately: the loss saturates at zero, training ends early, and the space has separation too thin to threshold reliably. Too large and almost no triplet clears it: gradients keep pushing even well-separated negatives, the model spends capacity spreading things it has already solved, and in the worst case it distorts or destabilises the space. Tune it against the metric you actually care about — verification error at your operating point — not against the training loss, because the training loss can be driven to zero by a margin that is simply too small to demand anything. ## What an interviewer is listening for Three things: the formula with the hinge; the word *gap* rather than *order*; and the observation that zero-loss triplets contribute no gradient, which is what makes triplet selection a first-class part of the method rather than a detail.