roadmap
- 01
- 02
- 03
- 04
- 05
- 06
- 07
- 08
Explain why a prompt works and how to make LLM output reliable, the way practitioners do in interviews. Covers LLM basics, prompt design and reasoning techniques, context, evaluation and safety.
311 of 311 questions in your plan: 0 known, 0 in progress, 0 due, 0 skipped, 311 new. 0 cut, outside your plan.
Few loops name a round after prompting. Its questions turn up in the loops of engineers who ship features on a language model: in a coding task whose model output another function must parse, in the design round once a retrieval or agent feature is on the whiteboard, and sometimes as a take-home that hands over a prompt failing on part of a sample and asks for a fix plus the evidence that it is one. Interviewers favor failures with a mechanism underneath them. A model miscounts the letters of a word it spells correctly, because it reads tokens, not letters. An instruction is obeyed at the end of a long document and ignored in its middle. Few-shot examples that share one label pull answers toward it. A JSON mode reply parses but breaks the schema. A timestamp at the top of a system prompt makes each request miss the cache. What sets levels apart is where the fix lands. Rewording until the example on screen passes is where most candidates begin. From middle level up, the answer starts with fixed test inputs and a score before the prompt is touched, and senior candidates know when to stop prompting: a field the business depends on gets a validator in code, a tool with side effects gets a permission check no retrieved text can talk past, and behavior no wording holds steady gets weighed against fine-tuning.
The plan opens beneath the prompt. [Tokenization](/topics/found-llms-tokenization) accounts for the bill and the strange failures, [context windows](/topics/found-llms-context-windows) for what becomes of material deep in a long input, and [the model's limits](/topics/found-llms-capabilities-limits) for invented citations and for why examples in a prompt teach anything at all. The module closes on [whether to fine-tune instead](/topics/found-fine-tuning-vs-prompting), worth settling before an afternoon goes into wording. [Prompt Design](/topics/found-prompt-engineering) is the craft proper, and later modules build on it. The module after it turns to what comes back: [decoding controls](/topics/found-llms-inference-sampling-decoding-strategies) such as temperature, then [structured output](/topics/found-ai-engineering-structured-output), where a schema is enforced while tokens are chosen or checked and repaired afterwards, with tool calls held to the same standard. Reasoning Techniques leans on both. [Self-consistency](/topics/found-chain-of-thought-self-consistency), inside [chain of thought](/topics/found-chain-of-thought), samples several chains and takes a vote, which makes sense once temperature does. [Tree of thought](/topics/found-tree-of-thought) turns reasoning into a search over scored partial states, and the [reason-act-observe loop](/topics/found-react-reasoning-loop) puts tool calls between thoughts, relying on the tool-call reliability just covered. Those loops lengthen the input each turn, and Context & Caching handles the cost. [Context engineering](/topics/found-context-engineering) decides what earns a place in the window and in what order; [prompt caching](/topics/found-prompt-caching) rewards a prompt whose stable part comes first. The last two modules swap judgement for measurement. [Automatic prompt engineering](/topics/found-automatic-prompt-engineering) searches over candidate prompts against a score. [Learned soft prompts](/topics/found-fine-tuning-soft-prompt-tuning) give up readable wording altogether: gradients train vectors in its place, which takes access to the model itself. [Evaluation](/topics/found-ai-engineering-evaluation) widens that score into a practice: a model as judge and where it misleads, a regression suite that gates a prompt change the way tests gate code. The plan ends on [prompt injection](/topics/found-ai-engineering-guardrails-safety-injection-mitigation), which better wording does not repair, since the attacker writes words too.
Days before a screen, put them into the first three modules, which hold the mechanism and structured-output questions most LLM loops reach, with importance trimmed to must know; tree search and compiled prompt programs can wait. Over months, build a test set rather than a prompt: one small task, such as pulling fields from messy support tickets, a few dozen inputs with known answers, and a score rerun after each technique. Because every later technique needs that set to prove itself, read [evaluation and iteration](/topics/found-prompt-engineering-evaluation-iteration) third inside Prompt Design, right after instructions and few-shot examples. Data scientists usually find the first module familiar and the gaps further down, in structured output, caching and injection; backend engineers who already call a model API tend to be the reverse. Vector stores, agent frameworks and the servers that host a model fall outside this plan, and fine-tuning appears only as the alternative to weigh, apart from learned soft prompts, the one trained form of prompt. The [AI Engineer roadmap](/roadmaps/ai-engineer) takes up the application built around the prompt. Attacking a model, rather than the defenses catalogued here, belongs to [AI Red Teaming](/roadmaps/ai-red-teaming).
Few loops name a round after prompting. Its questions turn up in the loops of engineers who ship features on a language model: in a coding task whose model output another function must parse, in the design round once a retrieval or agent feature is on the whiteboard, and sometimes as a take-home that hands over a prompt failing on part of a sample and asks for a fix plus the evidence that it is one.
Interviewers favor failures with a mechanism underneath them. A model miscounts the letters of a word it spells correctly, because it reads tokens, not letters. An instruction is obeyed at the end of a long document and ignored in its middle. Few-shot examples that share one label pull answers toward it. A JSON mode reply parses but breaks the schema. A timestamp at the top of a system prompt makes each request miss the cache.
What sets levels apart is where the fix lands. Rewording until the example on screen passes is where most candidates begin. From middle level up, the answer starts with fixed test inputs and a score before the prompt is touched, and senior candidates know when to stop prompting: a field the business depends on gets a validator in code, a tool with side effects gets a permission check no retrieved text can talk past, and behavior no wording holds steady gets weighed against fine-tuning.
The plan opens beneath the prompt. Tokenization accounts for the bill and the strange failures, context windows for what becomes of material deep in a long input, and the model's limits for invented citations and for why examples in a prompt teach anything at all. The module closes on whether to fine-tune instead, worth settling before an afternoon goes into wording.
Prompt Design is the craft proper, and later modules build on it. The module after it turns to what comes back: decoding controls such as temperature, then structured output, where a schema is enforced while tokens are chosen or checked and repaired afterwards, with tool calls held to the same standard.
Reasoning Techniques leans on both. Self-consistency, inside chain of thought, samples several chains and takes a vote, which makes sense once temperature does. Tree of thought turns reasoning into a search over scored partial states, and the reason-act-observe loop puts tool calls between thoughts, relying on the tool-call reliability just covered.
Those loops lengthen the input each turn, and Context & Caching handles the cost. Context engineering decides what earns a place in the window and in what order; prompt caching rewards a prompt whose stable part comes first.
The last two modules swap judgement for measurement. Automatic prompt engineering searches over candidate prompts against a score. Learned soft prompts give up readable wording altogether: gradients train vectors in its place, which takes access to the model itself. Evaluation widens that score into a practice: a model as judge and where it misleads, a regression suite that gates a prompt change the way tests gate code. The plan ends on prompt injection, which better wording does not repair, since the attacker writes words too.
Days before a screen, put them into the first three modules, which hold the mechanism and structured-output questions most LLM loops reach, with importance trimmed to must know; tree search and compiled prompt programs can wait.
Over months, build a test set rather than a prompt: one small task, such as pulling fields from messy support tickets, a few dozen inputs with known answers, and a score rerun after each technique. Because every later technique needs that set to prove itself, read evaluation and iteration third inside Prompt Design, right after instructions and few-shot examples.
Data scientists usually find the first module familiar and the gaps further down, in structured output, caching and injection; backend engineers who already call a model API tend to be the reverse.
Vector stores, agent frameworks and the servers that host a model fall outside this plan, and fine-tuning appears only as the alternative to weigh, apart from learned soft prompts, the one trained form of prompt. The AI Engineer roadmap takes up the application built around the prompt. Attacking a model, rather than the defenses catalogued here, belongs to AI Red Teaming.