How does balancing few-shot demonstrations across demographic groups reduce skew in an LLM's answers, and what did Si et al. measure?
answer
- shots teach a pairing
- group crossed with outcome
- one-sided either way stays biased
- 16 shots, gap 51 against 8
- an all-neutral set has its own price
basics
~20 sDemonstrations teach which answer goes with which group, so balancing pairs each group with each outcome equally. Si et al. (2022) measured a far smaller pronoun-bias gap with balanced shots than with one-sided ones, on one large model.
solid answer
~40 sFew-shot demonstrations teach by pattern, so a set that keeps showing one group with one answer teaches that pairing. Balancing means crossing group with outcome, so that neither predicts the other. Si et al. (2022) tested this on one large pretrained model. On a gender coreference benchmark with 16 shots, ambiguous pro-stereotype shots alone gave an accuracy gap of 51.0 points on ambiguous sentences, ambiguous anti-stereotype shots alone flipped it to -30.0, and balanced shots gave 8.1. So a one-sided set moves the skew rather than removing it, and balance shrinks it without reaching zero. On the BBQ benchmark, balanced shots gave an ambiguous bias score of 2.4 against about 24 for either one-sided set. All-neutral shots scored 0.0, but accuracy on answerable questions collapsed to 13.2%.
code
plaintext · 5 linesGroup Application evidence Answer
A income 3.0x payment, no missed payments yes
A income 1.1x payment, 2 missed payments no
B income 3.0x payment, no missed payments yes
B income 1.1x payment, 2 missed payments nogo deeper
Recall that a demonstration set can teach a pairing of group and answer, and that balancing means every group appears with every outcome.
Walk through the mechanism and the numbers: a one-sided set moves the gap, including flipping it negative, and balance shrinks it. Keep label balance and group balance apart.
Show that you would read the table critically: the effect concentrates on ambiguous inputs, balance was not smallest on every slice, and accuracy must be read beside the bias score.
Weigh the upkeep: a crossed shot grid grows quickly with attributes, so a lead decides whether to maintain one, strip group signals from shots, or stop relying on shots to carry this.
## Why demonstrations carry demographic skew A **demonstration** (or shot) is a worked input and answer placed in the prompt. The model infers the task from these, including patterns the author never meant to teach. If every approved applicant in the shots has one kind of name, the set teaches a pairing of group and outcome, and the model can apply it wherever the input leaves room for doubt. Si et al. (2022) put it directly: when the examples are biased towards certain groups, the model they studied picks up those biases. **Balancing** means crossing the two variables so that knowing a shot's group tells you nothing about its answer. It is not the same as balancing labels: a set of half yes and half no can still give every yes to one group. How many of each label to show, and what sequence to show them in, are separate effects covered with few-shot ordering and exemplar selection. ## What Si et al. measured All of this is one model, tested in 2022, on benchmark tasks. A later model may respond differently. **WinoBias** is a gender coreference benchmark: a sentence has a pronoun and an occupation, and the correct reading either fits a stereotype (pro-bias) or goes against it (anti-bias). The **bias gap** is accuracy on pro-bias sentences minus accuracy on anti-bias sentences. Type I sentences are ambiguous and need world knowledge; Type II can be resolved from syntax. Each row used 16 shots. | Demonstrations | Type I pro | Type I anti | Type I gap | Type II gap | |---|---|---|---|---| | Balanced, 4 from each of the four subsets | 89.2 | 81.1 | 8.1 | 3.7 | | All Type I pro-bias | 93.4 | 42.4 | 51.0 | 12.2 | | All Type I anti-bias | 50.8 | 80.8 | -30.0 | 6.3 | | All Type II pro-bias | 87.6 | 59.5 | 28.1 | 1.3 | | All Type II anti-bias | 85.5 | 68.2 | 17.3 | 0.5 | Four things to read out of this table: - **One-sided in the other direction is not a fix.** All anti-bias shots flipped the Type I gap to -30.0 instead of closing it. - **The damage concentrates where the input is ambiguous.** The authors attribute the stronger effect on Type I to its ambiguity, since Type II has clear syntax cues. - **Balance shrinks the gap and leaves some.** It was 8.1 on Type I, not zero. - **Balanced was not smallest on every slice.** On Type II, two single-subset sets gave smaller gaps (1.3 and 0.5), but they gave far larger gaps on Type I. The paper's "least biased" holds overall, not cell by cell. On **BBQ**, a question-answering bias benchmark, each row used 8 shots. Balanced shots gave an ambiguous bias score of 2.4 with 96.8% ambiguous accuracy and 76.0% accuracy on answerable questions. All pro-bias and all anti-bias sets on ambiguous questions gave scores of 24.7 and 23.6. A set of only ambiguous questions answered "unknown" scored 0.0, and the model learned to say unknown almost always: accuracy on answerable questions fell to 13.2%. ## Doing it on a screening prompt For a yes-or-no pre-screening prompt reading short loan applications, this is common practice rather than anything the paper prescribes: 1. **List the attributes the prompt can see,** including names, which signal group imperfectly. 2. **Cross them with the outcome.** With two groups and two outcomes, 8 shots give 2 per cell. With four race labels, three gender labels, three age bands and two outcomes there are 4 x 3 x 3 x 2 = 72 cells, which no prompt holds, so rotate attributes across shots and check each one's marginal balance. 3. **Mirror the evidence.** Use the same application facts for each group, so that the facts justify the answer and the group does not. 4. **Test both sides of the change.** Compare the swap gap and the accuracy on held-out applications, before and after. For free-text output such as a job-ad writer, there is no outcome to cross. Balance means varying who is portrayed, or writing shots that portray nobody, then counting the wording in a sample of outputs. ## What balance does not do Balancing removes what your own shots teach. It does not remove associations from training, and Si et al. call the mitigated model "still far from perfect". Balanced shots can also cost accuracy against one-sided ones on answerable questions (76.0% here against 97.3% for the pro-bias set), so a score on one axis is not enough to choose a prompt.
- Why not fill the set with shots answered 'unknown', since that scored 0.0 on BBQ?Because the model learned the pattern and said unknown nearly everywhere. Si et al. report 99.9% accuracy on ambiguous questions and 13.2% on answerable ones with that set. A zero score there records abstention, not fairness. A screening prompt that declines almost every application has no use, whatever its bias score.
- The attributes cross into more cells than the prompt can hold. What then?Rotate attributes across a small set of shots and check that each attribute on its own is balanced against the outcome, accepting that combinations go uncovered. Alternatively remove group signals from the shots, so they teach none at all. This is common practice rather than a measured result, so the swap test on held-out inputs decides.
- Using only anti-stereotype shots removes the bias
- Equal numbers of each group is enough, whichever answers they get
- Half yes and half no in the shots means the set is demographically balanced
- Balanced demonstrations bring the bias gap to zero
- Si et al. showed balanced shots win on every metric and slice