Prompt engineering

Dynamic exemplar retrieval picks the few-shot demonstrations per request, pulling the most similar labelled examples from a pool by vector similarity. A fixed block hardcodes the same demonstrations into the prompt for every request, whatever the input looks like.

Demonstrations define the label space the model treats as live. A class with no example is predicted far less often than it should be, even when the instruction names it, so coverage of every label comes before adding more examples.

Prompt chaining runs one task as an ordered series of model calls, each stage's output feeding the next stage's prompt. Every stage gets a narrow instruction and a checkable result, so a single step can be validated, retried, or replaced.

Escalate only when the cheaper rung measurably fails. Direct answers suit lookup and formatting; chain-of-thought pays on multi-step reasoning; sampling several chains and voting pays only when the answer is a short discrete value worth the multiplied cost.

Repeated hand-tuning fits a prompt to the examples you keep looking at. A held-out test set, scored rarely and never read during tuning, is the only honest estimate of how the prompt behaves on inputs you have not seen.

Wrong labels hurt far less than most people expect. On classification tasks, swapping gold labels for random ones drawn from the same label set often costs only a few points, while destroying the input-label format costs much more.

Order is not cosmetic: the model reads the demonstration sequence as evidence about which label is likely, and examples nearest the end pull hardest. The same four exemplars in different orders can move a sentiment task from the fifties into the high eighties.

Find it empirically: sweep the shot count on a held-out set and ship the first point where added examples stop paying. Gains typically flatten after a handful, while every extra example keeps costing tokens and latency on every request.

Delimiting untrusted text tells the model which span is material to work on rather than instructions to follow, which removes ambiguity and makes prompts assemblable by code. It enforces nothing, though: the model still reads one undifferentiated token stream.

A format instruction only biases the model; every token stays available, so it can still emit a preamble, a ```json fence, or a truncated object. Constraining decoding — a schema-based structured-output mode or a grammar — makes invalid output impossible rather than unlikely.

Fail closed: reject the response, never coerce or scrape it. Then optionally send one targeted repair prompt quoting the exact validator error, cap retries at one or two, and fall back to a deterministic path or human review while logging the failure rate.

Higher layers win. Provider policy outranks the application's system prompt, which outranks the end user's turn. Tool results and retrieved documents carry no instruction authority at all — they are data the model reasons about, not orders it follows.

Adherence decays because the standing rule sits at the far end of a growing history: it competes with thousands of tokens of newer, more concrete dialogue, and the model's own earlier replies — including any that already broke the rule — become the strongest example of how this conversation behaves.

Pin everything except the prompt: the eval dataset version, the model version, decoding parameters, and — if scoring is automated by a model — the judge and its rubric. Anything unpinned becomes an alternative explanation for every score move.

Prompt caching matches an exact token prefix from the start of the request. Retrieved exemplars differ on every call, so everything from that block onward is uncached — placed near the top, the entire prompt is reprocessed every single request.

Put a check at the seam. Validate the extracted fields against a schema, require each value to appear verbatim in the source, allow the stage to abstain rather than guess, and pass the source forward so the judging stage can disagree with the extraction.

A system prompt is guidance, not code. Nothing checks it at runtime, so a $100 cap written in prose holds only as often as the model chooses to hold it. The cap must be enforced in the refund tool or the payments service, which reject larger amounts regardless of what the model asks for.

Restate the rule close to where generation happens — a short constraint block appended after the newest user message — rather than rewriting the system prompt. Re-anchor only the two or three rules that actually drift, keep the wording byte-identical every turn, and enforce anything business-critical with a programmatic check instead.

Ask for reasoning inside one delimiter and the final answer inside another, then strip the reasoning before display. It is still generated and can be logged for review — hiding it is a rendering step, not a prompt shortcut.

Named XML-style tags suit prompts holding several distinct or nested blocks: each block gets a name and an explicit close marker you can refer to. Triple-backtick fences suit short literal or code spans. Consistency matters more than the vocabulary.

Ask for Markdown when a human reads the output directly, and JSON when code consumes it. Mixing them breaks parsing, since prose around an object stops the whole response from parsing. Put human-facing text inside a JSON field instead.

Remove one section at a time and rescore the identical eval items. Sections whose removal barely moves the score are not earning their tokens; sections that cause a clear drop are load-bearing. Then rebuild additively to catch redundant pairs.

The demonstrated shape usually wins over the written instruction. The model emits bare labels and drops the reason, or produces an unstable mix across calls. Fix the exemplars so every one carries both fields, in the order you want them produced.

Nearest neighbours are similar to the query, not representative of the task. For a borderline case they often all carry one label, or duplicate each other, so the prompt quietly argues for that one answer instead of showing the model where the decision boundary sits.

Feed the prompt an input carrying no task evidence, such as "N/A" or an empty string, and read the label probabilities it returns. Those probabilities are the prompt's built-in prior; dividing real predictions by them and renormalising removes most of the bias.

Both, in a specific order: cover the distinct modes of the traffic you actually receive, then weight roughly toward the common ones. Cloning one typical case wastes tokens; filling the set with exotic cases teaches a distribution your users do not send.

Least-to-most prompting asks the model to first list the subproblems a hard question contains, then solve them in dependency order, each answer feeding the next. The decomposition is produced at run time by the model, not authored in advance by an engineer.

It depends on why the constraint exists. For a stylistic preference, satisfy the user's intent as far as the constraint allows and say what was kept. For a constraint protecting someone other than the user, keep it and refuse the removal — briefly, without lecturing.

Segment the traffic. Reason only on the slice whose errors are genuinely multi-step; answer directly on lookup and formatting work, where forced deliberation adds latency and can talk the model out of a correct answer. Gate on a cheap difficulty signal and measure each segment separately.