How do LangChain few-shot prompt templates select which examples to include?

## The two moving parts A few-shot template is composed of a per-example template and a source of examples. `example_prompt` says how one example renders. For the string case that is a `PromptTemplate` such as `"Q: {input}\nA: {output}"`. For the chat case it is a small `ChatPromptTemplate.from_messages([("human", "{input}"), ("ai", "{output}")])`, so every example becomes two real messages. The example source is either `examples` (a list of dicts whose keys match the example prompt's variables) or `example_selector` (an object asked, per request, which examples to use). Supplying both is a configuration error; supplying neither leaves nothing to render. ## Static examples ``` few_shot = FewShotChatMessagePromptTemplate( example_prompt=example_prompt, examples=[{"input": "2+2", "output": "4"}, {"input": "2+3", "output": "5"}], ) final = ChatPromptTemplate.from_messages([ ("system", "You are a calculator."), few_shot, ("human", "{input}"), ]) ``` Note how the few-shot block nests inside a larger chat prompt: it is just another element that expands to multiple messages. In the string variant you instead get `prefix`, the rendered examples joined by `example_separator`, and `suffix` — the suffix being where the live input goes. Static examples have properties production likes. The rendered block is identical every call, so it sits inside the stable prompt prefix that providers cache and that observability tools fingerprint. It costs nothing at request time. It is trivially reviewable in a diff. And when the model misbehaves you can reproduce the exact prompt. Their weakness is coverage. If your inputs span several distinct shapes, a fixed handful of demonstrations either bloats the prompt to cover all of them or biases the model toward whichever shapes made the list. ## Dynamic selection An example selector implements a small interface: given the input variables of this call, return the examples to use. LangChain ships several, of which two matter most. `SemanticSimilarityExampleSelector` embeds each stored example and, at request time, embeds the incoming input and returns the k nearest. Construct it directly from a vector store, or with `from_examples(examples, embeddings, vectorstore_cls, k=2)`, which builds the store for you. This is the selector people mean when they say "dynamic few-shot": the model sees demonstrations that look like the question actually asked. `LengthBasedExampleSelector` takes as many examples as fit a length budget, dropping the rest. It is about token safety rather than relevance — useful when example sizes vary wildly. When a selector is used, the few-shot template's own `input_variables` must include the fields the selector needs, because the selector is called with the formatting inputs. ## The tradeoff an interviewer is listening for Dynamic selection is not free, and the costs are operational rather than conceptual: 1. **Latency and dependency.** Every request now embeds the input and queries a vector store before the model call. That is an extra network hop and an extra thing that can be slow or down, sitting in front of every generation. 2. **Cache destruction.** Provider prompt caching keys on a stable prefix. If the examples change per request and sit near the top of the prompt, nothing before the live input is ever reused. On a high-volume endpoint that can dominate the accuracy win in cost terms. Mitigation: put dynamic examples *after* the stable system content, or keep a static core set and add only one or two dynamic ones. 3. **Reproducibility.** "Why did it answer that?" now requires knowing which examples were selected, so the selected example ids belong in your trace alongside the prompt. 4. **Silent drift.** Adding examples to the pool changes behaviour for inputs you never tested, because a new neighbour can displace an old one. A fixed list changes only when someone edits it in a reviewed diff. ## Practical guidance Start static. A handful of well-chosen, diverse demonstrations is often within noise of a retrieval-based selector, and it is dramatically easier to operate. Move to a selector when you can show, on an evaluation set, that a fixed set cannot cover the input distribution — and then log which examples were chosen, cap k, and place the block where it does the least damage to your cached prefix. Also remember what few-shot examples are *for* in a chat model: with the chat variant they arrive as prior turns, which strongly shapes output format. If the goal is purely a rigid output shape, a structured-output constraint is usually a cheaper and more reliable lever than demonstrations.