What does in-context learning change in an LLM if no weights are updated?

## The claim in one line In-context learning (ICL) is the ability of a large language model to perform a task it was never specifically trained for, using only the instructions and worked examples placed inside its prompt. Nothing about the model changes. No optimizer step runs, no gradient is computed, and the parameters are identical before and after the call. The word "learning" is historical and slightly misleading; **conditioning** is the accurate description. ## What the model actually does with your examples A language model is a function from a sequence of tokens to a probability distribution over the next token. Everything in a request — the system instructions, the four labelled example claims you pasted, the new claim you want triaged — is one flat token sequence. During the forward pass, attention lets the position where the answer will be generated read every earlier position, so the content of the demonstrations shows up in the hidden state at that position, and therefore in the probabilities that come out. A token is sampled from that distribution, appended, and the process repeats. When the response ends, that working state is discarded. Providers often cache the internal state of a repeated prompt prefix so the next call with the same prefix is cheaper or faster. That is an infrastructure optimisation of the same computation — the cached state is derived entirely from tokens you sent — not a form of learning. The weights are still untouched. ## Why being pedantic about this pays off Three practical facts follow directly: 1. **Nothing carries over.** If an insurance claim-triage prompt produces the right label after you add a worked example, that improvement exists only in requests that carry the example. It is a property of your template, not of the model. 2. **Nothing leaks sideways.** Your examples do not change behaviour for other users or other endpoints of the same model. Statelessness is a safety property as much as a limitation. 3. **The prompt is the entire configuration.** To explain why a production answer came out the way it did, you need the exact text that was sent. Log it, version it like code, and treat a prompt edit as a deploy. ## What conditioning buys you Speed of iteration is the whole point. A claim-triage prompt that shows four adjudicated claims labelled `total_loss`, `repairable` and `fraud_review` fixes both the label vocabulary and the output shape immediately: the model now knows which strings are legal outputs and what a well-formed answer looks like. Changing that behaviour is a text edit, testable in seconds, revertible instantly, and variable per request — you can send a different example set to a different segment on the same model, which no weight-level change can do cheaply. ## What it costs The cost is recurring and structural: - **Tokens and latency.** Sixteen demonstrations are re-processed on every single call. At a million calls a day that is a permanent line item, whereas a training cost is paid once. - **Context budget.** Examples compete for the same window as the actual input, retrieved documents and conversation history. Long example blocks crowd out the material the answer actually depends on. - **Fragility.** Because the effect is purely conditioning, it is sensitive to how the examples are written, formatted and ordered. There is no artifact you can inspect to know what the model "knows"; there is only the prompt you sent. - **No sideways generalisation.** Fixing a failure in one prompt does not fix the same failure in another call site that happens to use the same model. ## The contrast to name in an interview Methods that update parameters produce a persistent change: the behaviour is baked in, per-request prompts get shorter, and the cost moves to a one-off training run plus the operational weight of owning a model version. In-context learning produces no persistent change and moves the cost into every request forever. Which one a given system should use is a separate decision with its own tradeoffs; the point here is that they are different *kinds* of change, and confusing them is the classic junior error. ## Common confusions worth pre-empting - **"The model learned my writing style."** No — the template that carries your style samples did. Delete the samples and the style disappears. - **"The examples fine-tune the model a little."** There is no gradient anywhere in a normal inference request. - **"Same prompt, same answer."** Conditioning reshapes a distribution; sampling still draws from it, so identical requests can differ unless randomness is constrained. - **"The model remembers what I told it earlier."** Only because your client re-sends the conversation. Statelessness is the default; any memory is something your system built around the model.