How does the APE method (Zhou et al., 2023) discover an instruction from examples?

## The problem APE poses Hand-tuning a prompt is a person guessing at phrasings and eyeballing a few outputs. APE (Zhou et al., 2023 — "Large Language Models Are Human-Level Prompt Engineers") reframes that as a search problem with three components: a *space* of natural-language instructions, a *proposal mechanism* that samples from it, and a *scoring function* over a dataset. The claim that made it notable was not that the search was clever — it is deliberately simple — but that an LLM is a good enough proposer that the simple search works. ## Step 1 — instruction induction The proposal step is inverse inference: given examples of the task's inputs and outputs, ask a model what instruction would have produced them. The published templates are conversational rather than technical — a forward-mode template of the shape "I gave a friend an instruction and several inputs. The friend read the instruction and wrote an output for every one of the inputs. Here are the input-output pairs: … The instruction was", and a reverse-mode variant that puts the blank in the middle for models trained to infill. This is the part that makes APE a *meta-prompting* method: the model is writing prompts, and the prompt that makes it do so is itself the object of design. Note what the induction step consumes: demonstrations, not a description. You need a labelled sample of the task, which is the same asset any honest evaluation needs. If you cannot produce twenty correct input-output pairs, you cannot run APE — and arguably you do not yet know what the task is. ## Step 2 — selection Each candidate instruction is executed on a scoring subset and ranked by an objective. The two APE reports are execution accuracy (run the instruction, compare the output against the gold label with a task-appropriate match) and log-probability of the target answer under the executing model, which gives a smoother signal on tasks where exact match is brittle. To keep cost sane the method filters aggressively: score all candidates on a small slice, discard the clearly bad ones, and only spend a large scoring budget on survivors. The non-negotiable detail is that the score comes from *running* the instruction with the model that will run it in production. An instruction that reads beautifully and scores badly is a bad instruction. This is the discipline that separates APE from prompt-writing advice. ## Step 3 — resampling The iterative variant takes the surviving high scorers and asks the model for variations that preserve meaning — a paraphrase-style meta-prompt of the shape "generate a variation of the following instruction while keeping the semantic meaning". Those variants are scored, the pool is re-ranked, and the cycle repeats. It is a local search around the current champion rather than a fresh draw from the whole space, which is why it converges quickly and why it can also converge on a local optimum. ## The result that made it famous APE's discovered zero-shot chain-of-thought trigger — "Let's work this out in a step by step way to be sure we have the right answer" — outperformed the human-written "Let's think step by step" on the arithmetic and reasoning benchmarks the paper evaluated. The finding is worth repeating in an interview not because that exact string is still the best trigger (it is model-specific and the models have all changed), but because of what it demonstrated: the mapping from phrasing to behaviour is not something human intuition reads accurately, and a scored search finds phrasings a person would not write. ## What APE assumes - **A labelled dataset exists** and is representative. Every downstream claim rests on the scoring set. - **The metric is trustworthy.** If the score is a proxy, the search will optimize the proxy; this is where most real APE deployments go wrong. - **The instruction is the bottleneck.** If the failure is missing knowledge, a bad retrieval step, or a model that cannot do the task at all, no phrasing recovers it. - **The executor is stable.** Prompts tuned against one model version can lose their edge when the model changes, so a tuned instruction is a versioned artifact with an expiry, not a permanent asset. ## Why it still matters As of mid-2026 nobody runs the 2023 code, but the formulation is the skeleton of everything that came after: propose, score against data, keep winners, iterate. When an interviewer asks about automatic prompt engineering, describing that loop and the assumptions it rests on is a stronger answer than naming a current tool, because the tools change and the loop does not.