Why add negative exemplars showing what a classifier must not flag?

## What positives alone actually teach Give a model six examples of documents that should be flagged and it will extract whatever those six have in common. That commonality is almost never the real rule. It might be a vocabulary, a document length, a phrasing style, or a source system — any correlate that happens to be shared. The model then applies that over-broad pattern to new input, and the visible symptom is a flood of false positives on things that superficially resemble the training examples. This is the quiet failure mode of exemplar selection: nothing looks wrong in the prompt. The examples are all correct. The set is simply one-sided, and one-sidedness is invisible until you look at the errors. ## Near-miss negatives do the work Not all negative examples are equally useful. A negative that is wildly unlike any positive — a random unrelated document — teaches almost nothing, because the model was never going to flag it. The valuable negatives are the **near misses**: inputs that share most surface features with a positive but land on the other side of the rule. In an export-control screening prompt, a useful negative is a shipment whose description uses the same technical vocabulary as a controlled item but describes a variant below the control threshold, or a destination that looks sensitive but is not on the restricted list. Each such pair — a positive and its near-miss negative — pins one dimension of the boundary. Three or four well-chosen pairs teach more than a dozen easy negatives. ## State the reason, not just the label A negative demonstration labeled only "not flagged" tells the model that this input is out, but leaves it to guess which feature was decisive. If the exemplar carries a short reason — naming the specific attribute that put it outside the rule — the model has a much better chance of applying the same distinction to a new case. This also has a review benefit: a human auditing the exemplar pool can check whether the stated reason matches the actual policy, which is far harder when the exemplar is a bare label. Keep the reasons in the same shape across all exemplars, positive and negative alike. Inconsistency between how positives and negatives are presented gives the model a shortcut — it can learn to predict the label from presentation rather than content. ## The over-correction risk Negatives are a correction, and corrections overshoot. A set that becomes negative-heavy pushes the model toward not flagging, which in a screening context is the more dangerous direction: a missed control violation costs far more than a false alarm a human clears in thirty seconds. So the balance is not "equal counts"; it is "whatever balance produces the error profile you want", and that requires an explicit position on the relative cost of the two error types. Practically: - Decide the acceptable false-negative rate first, since it is usually the constrained one. - Measure false positives and false negatives separately on a held-out set after every change to the exemplar set. A single accuracy number hides a swap of one error type for the other. - Change one thing at a time. Adding three negatives and rewording the instruction in the same commit makes the resulting shift uninterpretable. ## When negatives are not the right fix Over-flagging has other causes, and reaching straight for negative exemplars can paper over them: - **An ambiguous rule.** If two reviewers disagree about the correct label on borderline cases, the model cannot do better than the humans. Fix the definition and the labeling guidance first. - **A miscalibrated threshold.** If the system emits a score and flags above a cutoff, the cutoff may simply be wrong; that is a tuning change, not a prompt change. - **Mislabeled positives.** If some of the flagged examples should not have been flagged, the model is faithfully learning a wrong rule. Audit the positives before adding anything. ## What good looks like in an interview A strong answer names the mechanism (positives define a direction, not a boundary), specifies *near-miss* negatives rather than negatives in general, mentions carrying a stated reason, and closes with the measurement — tracking both error types, with an explicit stance on which one you are willing to trade. A weak answer just says "add counter-examples" and stops.