skip to content

Prompt engineering

7 roadmaps72 questionsupdated

Writing the instruction the model actually follows: how you structure the task, when examples help, how you pin down output format, and how you iterate with measurement instead of vibes. Interviewers treat this as the baseline skill for anyone building on LLMs.

on this pageshow

guide

overview

~1 min

Prompt engineering is the working skill behind any product that puts a language model in front of a task. Interviewers probe it less as wordcraft than as engineering discipline: can you tell a vague instruction from a precise one, explain why a prompt that worked on five inputs fails on the sixth, and show that a change actually helped rather than merely looked better on the example you had open. At senior levels the questions shift toward where a prompt stops being the right tool, and which guarantees have to live in code instead. The subject splits into six sections. [Instruction design](/topics/found-prompt-engineering-instruction-design) covers how the task is phrased, structured and broken into stages. [Few-shot prompting](/topics/found-prompt-engineering-few-shot-prompting), the largest, is teaching by demonstration, and how the choice, order and format of examples steer the result. [System and role prompts](/topics/found-prompt-engineering-system-role-prompts) deals with the layers of a chat request and what a standing instruction can and cannot hold. [Chain-of-thought](/topics/found-prompt-engineering-chain-of-thought) asks when making the model reason first is worth its tokens. [Output formatting](/topics/found-prompt-engineering-output-control) is about output a program can consume, and [evaluation and iteration](/topics/found-prompt-engineering-evaluation-iteration) is about measuring all of the above. Start with instruction design, since every other technique modifies a base instruction, then few-shot prompting. Take evaluation third rather than last: without it, the advice in every other section is untestable. The same idea often returns at a deeper level in senior questions, so revisit a section as your level rises.

primer

A few ideas sit under every section. Hold these and most of the questions below read as consequences rather than facts to memorise. - **The model continues text; it does not execute a contract.** The instruction, the examples, the conversation so far and any pasted document arrive as one sequence the model extends. Wording shifts probabilities rather than fixing outcomes, which is why a format request occasionally fails, why rules fade in long chats and why pasted content can behave like an order. - **What you show tends to outweigh what you say.** When demonstrations and the written instruction disagree, the demonstrations usually win. Examples define the answer shape, the set of labels in play and even the implied base rates, so a careless example set can quietly rewrite a careful instruction. - **Precise, structured and deliberately placed.** A good instruction names the task, the audience, the constraints and the output in concrete terms, marks off material from instructions, and sits where the model will weigh it. Many "the model is bad at this" complaints dissolve once the task is stated precisely or split into smaller steps. - **A prompt lowers a failure rate; code bounds it.** Anything where one violation counts as an incident, such as a spending limit, a permission or a format a machine must read, needs a check outside the model: a tool that refuses, a validator, constrained decoding. - **Every technique has a price.** More examples, reasoning steps, extra calls in a chain and repeated sampling each add tokens and latency to every request. The right amount is the smallest that measurably fixes a failure you have actually observed. - **Measure instead of eyeballing.** A prompt tuned against the handful of inputs you keep rereading is overfit to them. A fixed evaluation set, pinned conditions and a record of which prompt version produced which output turn prompt editing from opinion into engineering.

Zero-shot prompting
Asking for a task with an instruction alone and no worked examples; the baseline every added technique should be measured against.
Few-shot prompting
Including a small number of input-output demonstrations in the prompt so the model infers the task, the answer shape and the label set from them.
Exemplar
One demonstration in a few-shot prompt: an example input paired with the output the model should imitate.
Delimiter
A marker, such as a named tag or a fence, that separates parts of a prompt, typically the instructions from the material they operate on.
System prompt
The standing instruction an application places ahead of the conversation to set role, rules and tone; weighted heavily by the model but enforced by nothing.
Instruction hierarchy
The trained ordering a model uses to weigh conflicting instructions from different layers of a request, with retrieved and tool content meant to carry no authority.
Prompt injection
Text inside material the model processes, such as a document, web page or tool result, written to be read as instructions and redirect its behaviour.
Chain-of-thought
Prompting the model to produce intermediate reasoning before its final answer, which tends to help multi-step problems and adds cost everywhere else.
Self-consistency
Sampling several independent reasoning chains for one question and keeping the most common final answer, trading multiplied cost for accuracy on discrete answers.
Constrained decoding
Restricting which tokens the model may generate at each step, usually from a schema or grammar, so the output cannot break the required structure.
Label bias
A skew toward particular answers caused by the prompt itself, such as the mix, order or recency of labels in the examples, rather than by the input.
Held-out test set
Evaluation inputs set aside and not read during tuning, so their score estimates behaviour on unseen traffic rather than on the examples you tuned against.
Ablation
Removing one part of a prompt at a time and rescoring an unchanged evaluation set to learn which parts actually affect results.
LLM judge
A model that scores another model's outputs against a rubric; quick to scale, but itself a component to validate and pin.

The sections are layers of one prompt and one working method, not separate tricks. **Instruction design is the base layer.** Every other technique modifies an instruction that should already be clear. Delimiters and a stable section order are what later let examples, retrieved documents and conversation history be inserted by code without blurring into the instructions. Splitting a task into a chain is also where [output formatting](/topics/found-prompt-engineering-output-control) starts to matter, because every seam between stages is a place to parse and check. **Examples and instructions compete.** Demonstrations are a second channel of instruction, and the [demonstration format](/topics/found-prompt-engineering-few-shot-prompting-demonstration-format) section shows how that channel can override the first. Chain-of-thought often arrives through the same channel, as worked examples that show their reasoning, so the two sections share questions about format and cost. Choosing examples per request ties few-shot prompting to retrieval and to prompt caching. **Roles decide whose words count.** The [system and role prompts](/topics/found-prompt-engineering-system-role-prompts) section sorts everything above into layers: application rules, user turns, and pasted or retrieved content that should be treated as material. That is where prompt engineering meets security, and where the recurring answer is to move hard limits into tools and validators. **Evaluation closes the loop.** Every choice in the other sections, from how many examples and in which order to whether to reason first or whether to merge a chain, is an empirical question. [Evaluation and iteration](/topics/found-prompt-engineering-evaluation-iteration) supplies the method for answering it, which is why its ideas appear in senior questions across all six sections.

  1. Instruction Design →

    Every other technique edits a base instruction; learn to state a task precisely and keep material apart from instructions first.

  2. Few-Shot Prompting →

    How demonstrations teach the answer shape and label set, and how they can quietly contradict the instruction they accompany.

  3. Evaluation and Iteration →

    Learn to measure a change before collecting more techniques; later choices about examples, reasoning and chains depend on it.

  4. System and Role Prompts →

    The layers of a chat request, and why a standing instruction is guidance rather than a guarantee.

  5. Output Formatting →

    Needed as soon as code reads the answer: schemas, validation, and what happens when output fails to parse.

  6. Chain-of-Thought →

    Last, because deciding when reasoning pays relies on the cost awareness and measurement habits the earlier sections build.

  • Treating a system prompt as a security boundary; a limit written in prose holds only as often as the model complies, so enforce it in the tool or service.

  • Pasting untrusted text into a prompt unmarked, so the model cannot tell material to process from instructions to follow; marking it helps but does not make it safe.

  • Writing a careful instruction, then supplying examples that show a different answer shape; the examples usually win.

  • Leaving a label out of the few-shot set, or giving one label most of the examples, and then blaming the model for a skewed classifier.

  • Trusting a format instruction to produce parseable output and then scraping or coercing whatever came back, instead of validating it and rejecting failures.

  • Adding a step-by-step reasoning trigger to every request; on lookup and formatting work it adds latency and cost and can make answers worse.

  • Declaring a prompt better after rereading the same few inputs, with no held-out set and with the model version or sampling settings left unpinned.

Most senior questions in this hub are one of these choices in disguise, and naming the one you are making is often most of the answer. - **Describing versus demonstrating.** A written rule is cheap and transparent; examples convey nuance a description misses, but they cost tokens on every request and can teach rules you never intended. - **One call versus a chain.** A single call is faster and simpler to operate; a chain gives each step a narrow job and a place to check it, at the price of latency, more moving parts and errors that travel between stages. - **Fixed versus retrieved examples.** A fixed block is predictable and friendly to prompt caching; picking examples per request fits the input better but makes behaviour depend on the pool's contents and freshness. - **Reasoning versus a direct answer.** Asking for reasoning helps multi-step problems and costs tokens and latency everywhere else, so the decision belongs to a segment of traffic, not to the whole system. - **Prompt versus code.** Wording lowers a failure rate; a validator, a tool check or constrained decoding bounds it. The more a failure costs, the further its guard belongs from the prompt. - **Strict versus expressive output.** A tight schema makes output easy to consume; too tight, and it can force the model to invent a value or choose a wrong option rather than say it does not know.

Several shapes recur across the sections under different names; spotting one is often the quickest way into an unfamiliar question. - **Separate material from instructions.** Delimiters around documents, a fixed prompt skeleton, and tool or retrieved content treated as data rather than orders are the same idea at the text, template and role level. - **Check every seam.** Between chain stages, after structured output, and at the entrance to an example pool: validate what crosses the boundary and decide in advance what happens when it fails. - **Restate near the point of generation.** Rules that fade over a long conversation or a long output are repeated close to where the model is writing, not only at the top. - **Escalate only on measured failure.** Instruction alone, then examples, then reasoning, then sampling and voting: each rung is climbed when the cheaper one is shown to fall short. - **Change one variable, rescore the same set.** Ablations, shot-count sweeps, order searches and chain-versus-single-call comparisons are one experimental design pointed at different variables.

explore

report an issue with this guide →

questions

72 · 10 sections

What is prompt chaining, and why split a task across several model calls?

level: juniorimportance: must knowfreq 62%
basics
~20 s

Prompt chaining runs one task as an ordered series of model calls, each stage's output feeding the next stage's prompt. Every stage gets a narrow instruction and a checkable result, so a single step can be validated, retried, or replaced.

open as a page

Why delimit untrusted document text in a prompt, and why is that not a security control?

level: middleimportance: must knowfreq 72%
basics
~20 s

Delimiting untrusted text tells the model which span is material to work on rather than instructions to follow, which removes ambiguity and makes prompts assemblable by code. It enforces nothing, though: the model still reads one undifferentiated token stream.

open as a page

A four-stage screening chain extracts fields then judges inclusion; how do you stop stage-2 errors poisoning stage 3?

level: seniorimportance: must knowfreq 55%
basics
~20 s

Put a check at the seam. Validate the extracted fields against a schema, require each value to appear verbatim in the source, allow the stage to abstain rather than guess, and pass the source forward so the judging stage can disagree with the extraction.

open as a page

In a prompt, when should you use XML-style tags instead of triple-backtick fences?

level: juniorimportance: should knowfreq 55%
basics
~20 s

Named XML-style tags suit prompts holding several distinct or nested blocks: each block gets a name and an explicit close marker you can refer to. Triple-backtick fences suit short literal or code spans. Consistency matters more than the vocabulary.

open as a page

What is least-to-most prompting, and how does it differ from a fixed pipeline?

level: middleimportance: should knowfreq 45%
basics
~20 s

Least-to-most prompting asks the model to first list the subproblems a hard question contains, then solve them in dependency order, each answer feeding the next. The decomposition is produced at run time by the model, not authored in advance by an engineer.

open as a page

Why must few-shot examples use the same field names and delimiters as the live query?

level: juniorimportance: must knowfreq 65%
basics
~20 s

Demonstrations teach by pattern continuation. If the examples use one separator and field set but the live query arrives in a different shape, the query stops reading as the next item in the series, and the model's output format drifts or it keeps writing examples.

open as a page

What is dynamic exemplar retrieval in few-shot prompting, and how does it differ from a fixed example block?

level: juniorimportance: must knowfreq 55%
basics
~20 s

Dynamic exemplar retrieval picks the few-shot demonstrations per request, pulling the most similar labelled examples from a pool by vector similarity. A fixed block hardcodes the same demonstrations into the prompt for every request, whatever the input looks like.

open as a page

When picking few-shot exemplars, why must every label class appear?

level: juniorimportance: must knowfreq 58%
basics
~20 s

Demonstrations define the label space the model treats as live. A class with no example is predicted far less often than it should be, even when the instruction names it, so coverage of every label comes before adding more examples.

open as a page

In few-shot prompting, do wrong labels in the demonstrations hurt accuracy?

level: middleimportance: must knowfreq 55%
basics
~20 s

Wrong labels hurt far less than most people expect. On classification tasks, swapping gold labels for random ones drawn from the same label set often costs only a few points, while destroying the input-label format costs much more.

open as a page

Why can permuting the same four few-shot examples swing accuracy by double digits?

level: middleimportance: must knowfreq 62%
basics
~20 s

Order is not cosmetic: the model reads the demonstration sequence as evidence about which label is likely, and examples nearest the end pull hardest. The same four exemplars in different orders can move a sentiment task from the fifties into the high eighties.

open as a page

In an LLM chat request, which instruction layer wins: system, user, or tool output?

level: middleimportance: must knowfreq 66%
basics
~20 s

Higher layers win. Provider policy outranks the application's system prompt, which outranks the end user's turn. Tool results and retrieved documents carry no instruction authority at all — they are data the model reasons about, not orders it follows.

open as a page

Why does system-prompt adherence decay over a long conversation?

level: middleimportance: must knowfreq 62%
basics
~20 s

Adherence decays because the standing rule sits at the far end of a growing history: it competes with thousands of tokens of newer, more concrete dialogue, and the model's own earlier replies — including any that already broke the rule — become the strongest example of how this conversation behaves.

open as a page

In an LLM support agent, why can't a system prompt enforce a $100 refund cap?

level: seniorimportance: must knowfreq 72%
basics
~20 s

A system prompt is guidance, not code. Nothing checks it at runtime, so a $100 cap written in prose holds only as often as the model chooses to hold it. The cap must be enforced in the refund tool or the payments service, which reject larger amounts regardless of what the model asks for.

open as a page

How do you re-anchor a drifting system-prompt constraint mid-conversation?

level: seniorimportance: must knowfreq 55%
basics
~20 s

Restate the rule close to where generation happens — a short constraint block appended after the newest user message — rather than rewriting the system prompt. Re-anchor only the two or three rules that actually drift, keep the wording byte-identical every turn, and enforce anything business-critical with a programmatic check instead.

open as a page

A system prompt says "always cite sources" and the user says "no citations" — what should the assistant do?

level: middleimportance: should knowfreq 48%
basics
~20 s

It depends on why the constraint exists. For a stylistic preference, satisfy the user's intent as far as the constraint allows and say what was kept. For a constraint protecting someone other than the user, keep it and refuse the removal — briefly, without lecturing.

open as a page

When do you escalate from a direct answer to chain-of-thought or self-consistency?

level: middleimportance: must knowfreq 72%
basics
~20 s

Escalate only when the cheaper rung measurably fails. Direct answers suit lookup and formatting; chain-of-thought pays on multi-step reasoning; sampling several chains and voting pays only when the answer is a short discrete value worth the multiplied cost.

open as a page

How do you keep chain-of-thought reasoning out of the user-facing answer?

level: juniorimportance: should knowfreq 58%
basics
~20 s

Ask for reasoning inside one delimiter and the final answer inside another, then strip the reasoning before display. It is still generated and can be logged for review — hiding it is a rendering step, not a prompt shortcut.

open as a page

Chain-of-thought on every request costs tokens — how do you decide which ones need it?

level: seniorimportance: should knowfreq 48%
basics
~20 s

Segment the traffic. Reason only on the slice whose errors are genuinely multi-step; answer directly on lookup and formatting work, where forced deliberation adds latency and can talk the model out of a correct answer. Gate on a cheap difficulty signal and measure each segment separately.

open as a page

How much explicit chain-of-thought prompting is redundant on reasoning models?

level: principalimportance: should knowfreq 38%
basics
~20 s

Most of the trigger phrasing is. Models that deliberate internally already produce a chain, and long hand-written reasoning scaffolds can cut across it. What still pays is task-specific direction — what to check, in what order, and what the answer must contain.

open as a page

In an LLM prompt, why is "respond only with JSON" unreliable, and what beats it?

level: middleimportance: must knowfreq 78%
basics
~20 s

A format instruction only biases the model; every token stays available, so it can still emit a preamble, a ```json fence, or a truncated object. Constraining decoding — a schema-based structured-output mode or a grammar — makes invalid output impossible rather than unlikely.

open as a page

When an LLM's JSON fails schema validation, how should the calling code respond?

level: middleimportance: must knowfreq 63%
basics
~20 s

Fail closed: reject the response, never coerce or scrape it. Then optionally send one targeted repair prompt quoting the exact validator error, cap retries at one or two, and fall back to a deterministic path or human review while logging the failure rate.

open as a page

When should an LLM return Markdown instead of JSON, and why not mix both?

level: juniorimportance: should knowfreq 52%
basics
~20 s

Ask for Markdown when a human reads the output directly, and JSON when code consumes it. Mixing them breaks parsing, since prose around an object stops the whole response from parsing. Put human-facing text inside a JSON field instead.

open as a page

Why does an LLM's long structured output drift, like row 60 of a 100-row table losing a field?

level: seniorimportance: should knowfreq 44%
basics
~20 s

Adherence decays over a long generation: the format spec recedes into distant context, each new row is copied from nearby rows rather than the schema, and one omission propagates. Fix by batching, per-row validation with row-count postconditions, and constrained decoding.

open as a page

When does forcing a strict JSON schema on an LLM hurt output quality?

level: principalimportance: should knowfreq 36%
basics
~20 s

When the schema makes truth unrepresentable or removes reasoning room. Required non-nullable fields push the model to fabricate, a closed enum forces a wrong nearest label, and constraining from the first token denies it space to think before committing.

open as a page

Why hold out a separate test set when iterating on a prompt by hand?

level: middleimportance: must knowfreq 62%
basics
~20 s

Repeated hand-tuning fits a prompt to the examples you keep looking at. A held-out test set, scored rarely and never read during tuning, is the only honest estimate of how the prompt behaves on inputs you have not seen.

open as a page

What must a prompt regression suite pin so a score change is attributable?

level: seniorimportance: must knowfreq 55%
basics
~20 s

Pin everything except the prompt: the eval dataset version, the model version, decoding parameters, and — if scoring is automated by a model — the judge and its rubric. Anything unpinned becomes an alternative explanation for every score move.

open as a page

How would you ablate a 900-token prompt to find its load-bearing instructions?

level: middleimportance: should knowfreq 42%
basics
~20 s

Remove one section at a time and rescore the identical eval items. Sections whose removal barely moves the score are not earning their tokens; sections that cause a clear drop are load-bearing. Then rebuild additively to catch redundant pairs.

open as a page

How does failure-mode analysis of wrong outputs drive the next prompt revision?

level: seniorimportance: should knowfreq 46%
basics
~20 s

Read the wrong outputs, group them by hand into a few named failure modes, and count each. The biggest group sets the next revision's target, and each mode implies a different edit — or tells you the fix is not a prompt edit at all.

open as a page

How would you manage production prompts as versioned artifacts across a team?

level: principalimportance: should knowfreq 34%
basics
~20 s

Treat each prompt as code: a reviewed file under version control, an identifier recorded with every production response, an eval run attached to each change, a named owner, and a rollback that is one deploy away.

open as a page

What is step-back prompting, and how do its abstraction and reasoning steps work?

level: middleimportance: should knowfreq 35%
basics
~20 s

Step-back prompting first has a model answer a more general question about the concept or principle behind a task, then answer the original question grounded on those facts. The two steps are named abstraction and reasoning.

open as a page

In step-back prompting, how do you write the step-back question and the few-shot exemplars that teach the abstraction?

level: middleimportance: should knowfreq 20%
basics
~20 s

Strip the specifics from the question and ask about the concept covering them, so the answer still holds what the original needs. Exemplars pair an original question with its abstraction; one sufficed on the paper's tested tasks.

open as a page

How does step-back prompting differ from chain of thought and from decomposing a question into sub-questions?

level: seniorimportance: should knowfreq 30%
basics
~10 s

Chain of thought reasons forward from the question; step-back first climbs to a general principle, then reasons from it. Decomposition goes down into sub-questions, one-to-many, while many specific questions can share one abstraction, many-to-one.

open as a page

A step-back prompt still misses a fifth of your held-out questions: which step do you suspect first, and how do you confirm it?

level: seniorimportance: should knowfreq 25%
basics
~20 s

Suspect the reasoning step first. Zheng et al. (2023) place over 90% of step-back's physics errors there, with wrong principles a small fraction. Confirm by tagging failures per step and rerunning them with a known-correct principle.

open as a page

Across the tasks its paper tested, where did step-back prompting gain accuracy and where did it add little?

level: seniorimportance: should knowfreq 25%
basics
~20 s

In Zheng et al. (2023), one large model gained on physics, chemistry, time-sensitive facts and multi-hop questions. Step-back only tied chain of thought on arithmetic word problems and stayed flat without retrieval on a binary strategy benchmark.

open as a page

What is a prompt template, and how does it differ from the rendered prompt a model actually reads?

level: juniorimportance: must knowfreq 40%
basics
~20 s

A prompt template is fixed text with named placeholders; filling them with per-request values produces the rendered prompt. The model reads only the rendered prompt, so that text, not the template, is what you inspect and evaluate.

open as a page

How should a prompt template's render step handle typed inputs, and missing or empty values?

level: middleimportance: should knowfreq 34%
basics
~20 s

Check a prompt's values against a declared contract before filling anything. A missing required value fails loudly with no model call; a legitimately empty optional value drops its whole section, label included, so no dangling line reaches the model.

open as a page

Why log a prompt template's id and version beside the rendered text, and why must an eval render through production code?

level: seniorimportance: should knowfreq 22%
basics
~20 s

A prompt template's id and version group calls by source and let you compare changes; the rendered text shows what the model actually read. An eval must call the production render, or it scores strings production never sends.

open as a page

In a prompt template pipeline, why would a ticket body containing `{plan_tier}` reach the model with the real tier filled in, and how do you fix it?

level: seniorimportance: should knowfreq 27%
basics
~20 s

A customer value was rendered twice: a later pass scanned earlier output as template source. Compose trusted template text first, then fill values once in a single scan, and never hand an untrusted string to a template engine as source.

open as a page

Should three product teams share one prompt template for support replies or keep per-team copies, and who may edit a shared one?

level: principalimportance: nice to knowfreq 15%
basics
~20 s

The choice depends on whether the teams share a task. While they do, share one core with declared team values, a named owner and every team's eval gating edits. Split once team-specific branches dominate it.

open as a page

What is prompt ensembling, and how does it differ from sampling one prompt many times?

level: middleimportance: should knowfreq 35%
basics
~20 s

Prompt ensembling runs several deliberately different prompts on one input, such as reworded instructions or separate demonstration sets, then combines their answers. Sampling one prompt repeatedly varies only the decoding, while ensembling varies what the model is asked.

open as a page

In a prompt ensemble, when does weighting beat a plain majority on labels, and what stands in for a vote on free-text answers?

level: seniorimportance: should knowfreq 20%
basics
~20 s

Weights can help when prompts differ widely in accuracy or duplicate each other, though published results are mixed, so plain majority is a sound start. Free text needs reducing to countable fields or source spans, or an aggregator prompt.

open as a page

Three reworded prompts on an eight-label ticket classifier agree on the same wrong label: why can no vote fix that, and how do you measure it?

level: seniorimportance: should knowfreq 30%
basics
~20 s

A vote can only change inputs where prompts disagree, so a shared wrong label survives any vote or weighting. Measure on a held-out set: disagreement rate, unanimous-wrong rate, and pairwise overlap of error sets against an independent-errors baseline.

open as a page

Your three-prompt ticket ensemble gains half a point at triple the calls: when would you pay for it rather than improve one prompt or add a verifier?

level: principalimportance: should knowfreq 25%
basics
~20 s

Pay only when a large held-out run confirms the gain, a wrong decision costs more than the extra calls, and latency allows. Otherwise improve the prompt first, since it reaches unanimous errors a vote never can, at one call.

open as a page

In demonstration ensembling (DENSE), how are demonstrations split into buckets, and how are the buckets' predictions combined?

level: middleimportance: nice to knowfreq 15%
basics
~20 s

DENSE partitions the demonstrations into non-empty buckets, runs one prompt per bucket on the same input, and merges each bucket's label probabilities with a product, a mixture or a maximum, optionally weighting buckets by similarity to the input.

open as a page

What is prompt debiasing, and which two parts of an LLM prompt does it change to reduce demographic skew?

level: juniorimportance: should knowfreq 34%
basics
~20 s

Prompt debiasing rewrites a prompt to reduce demographic skew in model output, through two levers: demonstrations balanced across groups and an explicit fairness instruction. It can lower measured skew but proves nothing alone, so the output must still be tested.

open as a page

How does balancing few-shot demonstrations across demographic groups reduce skew in an LLM's answers, and what did Si et al. measure?

level: middleimportance: should knowfreq 22%
basics
~20 s

Demonstrations teach which answer goes with which group, so balancing pairs each group with each outcome equally. Si et al. (2022) measured a far smaller pronoun-bias gap with balanced shots than with one-sided ones, on one large model.

open as a page

What is a counterfactual input swap test for an LLM prompt, and how do you run one on a yes-or-no screening prompt?

level: middleimportance: should knowfreq 30%
basics
~20 s

A counterfactual swap sends the same input twice with only a name or stated group changed and compares the answers. On a yes-or-no screen, compare the probability of yes across many applications; a gap beyond noise traces to that field.

open as a page

How much can a fairness instruction in an LLM prompt reduce demographic bias, and why does the measured effect depend on the model?

level: seniorimportance: should knowfreq 26%
basics
~20 s

A fairness instruction can cut measured bias sharply on large models: one study reports 84% on one benchmark. Smaller models often did not respond, and other work found little effect, so re-measure it per model and task.

open as a page

An LLM prompt's score on the BBQ bias benchmark drops to near zero. Why might that not be good news for your own task?

level: seniorimportance: nice to knowfreq 16%
basics
~20 s

A BBQ score can fall because the model simply answers 'unknown' more often, which also costs accuracy on answerable questions. BBQ also tests short US-English question-answering contexts, so a low score says little about your own prompt and task.

open as a page