With Ollama 0.35.0, what context length does a model get when nothing sets one, and how do you change it?
answer
- depends on the machine
- total VRAM at server start
- 4k, 32k or 256k tiers
- OLLAMA_CONTEXT_LENGTH or num_ctx
- check the CONTEXT column
basics
~20 sOllama 0.35.0 picks the default from total VRAM at startup: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, 256k at 48 GiB and up. OLLAMA_CONTEXT_LENGTH changes it server-wide and options.num_ctx per request.
solid answer
~40 sIn Ollama 0.35.0 the default context scales with the GPU memory the server detects at start: 4096 tokens under 24 GiB, 32768 from 24 to 48 GiB, and 262144 at 48 GiB and above, summed across GPUs. `OLLAMA_CONTEXT_LENGTH` on the server replaces that default for every model, a model can carry its own `num_ctx`, and a native request can send `options.num_ctx`; the most specific one wins, and the result is capped at the model's trained limit. Agents and coding tools usually need far more than 4k. Because `num_ctx` is a load-time option, a request with a different value reloads the model, and a bigger context costs memory — so check the CONTEXT and PROCESSOR columns of `ollama ps` after changing it.
code
bash · 12 lines# Server-wide default for every model that does not set its own
OLLAMA_CONTEXT_LENGTH=64000 ollama serve
# Per request on the native route (a different value reloads the model)
curl http://localhost:11434/api/chat -d '{
"model": "your-model",
"messages": [{"role": "user", "content": "Summarise this file"}],
"options": { "num_ctx": 32768 }
}'
# Confirm what was actually allocated
ollama psgo deeper
Recall that Ollama 0.35.0's default context depends on VRAM, and that OLLAMA_CONTEXT_LENGTH or num_ctx changes it.
Explain the three VRAM tiers, the order in which request, model and server settings win, the cap at the trained limit, and that num_ctx is fixed when the model loads.
Spot the failure modes: agents truncated at 4k, a 256k default pushing layers to CPU, and clients with different num_ctx values forcing reloads; fix them with one server-wide value.
Standardise context per workload across a team's machines instead of accepting a hardware-dependent default, so the same model behaves the same on a laptop and a workstation.
## The default: chosen from VRAM at startup **Context length** is the maximum number of tokens a loaded model can hold in memory at once — the prompt, the conversation so far and the answer being generated. In Ollama 0.35.0, when nothing sets it, the server picks it from how much GPU memory it found when it started: | Total VRAM detected | Default context | |---|---| | under 24 GiB | 4096 tokens (4k) | | 24 to 48 GiB | 32768 tokens (32k) | | 48 GiB and up | 262144 tokens (256k) | A few details matter in practice: - The total is **summed across all detected GPUs**, minus any per-GPU reserve set with `OLLAMA_GPU_OVERHEAD`, and the code uses slightly lower cut-offs (23 and 47 GiB) so a card that reports a little under 24 GiB still lands in the 32k tier. - The choice is made **once, at server start**; the log records it as a "vram-based default context" line. - A requested context larger than the model was trained for is **capped** at the model's own limit, with a warning in the log. - If a load with an automatically chosen context runs out of memory, the scheduler retries **once** with the next smaller automatic context (32k from 256k, 4k from 32k). - Cloud models are set to their maximum context by default. Two lines in the documentation still print older figures: the Modelfile reference's parameter table and one FAQ answer. The context-length page and the server code are the current behaviour. ## Why the default is often wrong for you A 4k context on a 16 GiB laptop GPU keeps a model fast and fully on the GPU, but agents, coding tools and retrieval over long documents need far more; the documentation suggests at least 64000 tokens for such tasks. The model only sees what fits in its window, so an application that sends more than that loses the earliest part of its prompt — often the instructions. On a 48 GiB machine the opposite problem appears: a 256k default reserves a large amount of memory that a short chat never uses, which can push layers off the GPU. ## How to change it The settings stack from the most specific to the least: 1. **Per request** — `options.num_ctx` on the native `/api/chat` or `/api/generate` routes. 2. **Per model** — a `PARAMETER num_ctx` line baked into a model with a Modelfile (the Modelfile material covers how). 3. **Per server** — `OLLAMA_CONTEXT_LENGTH` set in the server's environment, which replaces the VRAM-based default for every model that does not set its own. 4. **The VRAM tier** — used only when none of the above is set. In an interactive `ollama run` session, `/set parameter num_ctx 8192` changes it for that session, and the desktop app has a context-length slider in its settings. The OpenAI-compatible routes have no field for it, which is why a server-wide or per-model setting is the usual answer for SDK-based clients. ## Changing it has a cost: reloads and memory `num_ctx` is a **load-time** option: the memory for the context is allocated when the model is loaded. If a request arrives with a different `num_ctx` from the one the model is loaded with, Ollama unloads and reloads the model with the new value before answering. Two clients that send different values to the same model therefore make it reload back and forth, and every reload costs seconds. A larger context also costs memory in proportion to its length; the arithmetic of that cache belongs to the inference foundations, but the symptom on this runtime is a CPU/GPU split in `ollama ps`. ## Verifying what you got Run `ollama ps` and read the **CONTEXT** column: it shows the context the model was actually loaded with, after defaults, overrides and caps. Check **PROCESSOR** on the same row — the right context is the largest one your clients need that still reads `100% GPU`. ## Picking a number - **Short chat or classification** rarely needs more than a few thousand tokens; the 4k tier is often fine. - **Agents, coding tools and web search** carry long instructions and tool output; the documentation recommends at least 64000 tokens. - **Retrieval over documents** needs room for every retrieved chunk plus the question and the answer, so size it from the retrieval settings, not from the model's maximum. - **The model's trained limit** is a ceiling, not a target: asking for it on a small GPU reserves memory a short request never uses. Whatever you choose, set it in one place so every client loads the model the same way.
- Two internal tools call the same Ollama model, one sending `num_ctx` 8192 and the other nothing. What do you see?The model reloads whenever the two alternate, because `num_ctx` is fixed at load time and the second tool gets the server default, which differs from 8192. Each reload costs seconds. Pick one value and set it server-wide with `OLLAMA_CONTEXT_LENGTH`, or in the model itself, so both tools load it the same way.
- A machine has two 12 GiB GPUs. Which default tier does Ollama 0.35.0 use?Usually the 32k tier. The server sums the memory of all detected GPUs, so two 12 GiB cards count as about 24 GiB, above the 23 GiB cut-off the code uses for the 24 to 48 GiB band. Setting `OLLAMA_CONTEXT_LENGTH` overrides that if 32k is more than the workload needs.
- Ollama gives every model a 2048-token context by default
- The default context is always the model's full trained length
- Changing num_ctx between requests is free
- OLLAMA_NUM_PARALLEL sets the context length
- A larger num_ctx than the model supports extends the model
Assumes Ollama 0.35.0, where the default context is chosen from detected VRAM (4k/32k/256k) instead of one fixed value for every machine.