skip to content

Ollama

Ollama runs models behind a daemon on port 11434, with an ollama.com library, Modelfiles and native plus OpenAI- and Anthropic-style APIs. Interviewers use it to test local-first judgement.

part ofLocal LLM runtimesread it →
on this pageshow

explore

questions

20

In Ollama, what does the PROCESSOR column of `ollama ps` tell you, and what does a 48%/52% CPU/GPU reading mean?

level: juniorimportance: must knowfreq 24%

answer

  1. loaded models, not pulled ones
  2. memory placement, not utilisation
  3. size against size_vram
  4. some layers left in system RAM
  5. CONTEXT column sits beside it

basics

~20 s

ollama ps lists the models currently loaded, and PROCESSOR says where each one's memory sits: 100% GPU is fully in VRAM, while 48%/52% CPU/GPU means about half stayed in system RAM, so those layers run on the CPU and generation slows sharply.

solid answer

~50 s

`ollama ps` shows the models loaded in memory right now — not everything pulled, which is `ollama ls` — with NAME, ID, SIZE, PROCESSOR, CONTEXT and UNTIL. PROCESSOR comes from two numbers in `/api/ps`: the loaded model's `size` and the part in GPU memory, `size_vram`. `100% GPU` means everything is in VRAM, `100% CPU` means nothing is, and `48%/52% CPU/GPU` means 48% of the loaded size sits in system RAM. That size includes the memory reserved for the context, not just the weights. A split is a big slowdown, because the layers left on the CPU run at system-memory speed and every token passes through them. The fix is to make the load fit — a smaller context, fewer parallel slots, another loaded model stopped, or a smaller tag — and then check that it reads `100% GPU`.

code

console · 4 lines
console
$ ollama ps
NAME             ID              SIZE      PROCESSOR          CONTEXT    UNTIL
your-model:14b   1a2b3c4d5e6f    15 GB     41%/59% CPU/GPU    32768      4 minutes from now
your-embedder    6f5e4d3c2b1a    849 MB    100% GPU           8192       2 minutes from now

go deeper

for a junior

Know that ollama ps shows loaded models, that PROCESSOR is where their memory sits, and that 100% GPU is the reading you want.

for a middle

Explain how the split is derived from size and size_vram, that the size includes context memory, and which settings make a model fit again.

for a senior

Diagnose a split from the whole row: compare CONTEXT with what clients need, count other loaded models, and confirm 100% GPU after each change rather than guessing.

for a principal

Treat 100% GPU as a service-level condition on shared boxes: decide which models and contexts a machine is sized for, and monitor size against size_vram instead of waiting for complaints.

## What `ollama ps` lists `ollama ps` asks the running Ollama server which models are **loaded in memory right now** — not which models are on disk. Pulled models are listed by `ollama ls`; a model only appears in `ollama ps` after a request (or a preload) has loaded it, and it disappears again when it is unloaded. The same data is available over HTTP from `GET /api/ps`, which is what the command calls. Each row has six columns: - **NAME** and **ID** — the model tag and the first characters of its digest. - **SIZE** — how much memory the loaded model occupies. This is the size of the *running* model, not of the file: it covers the weights plus the memory reserved for the context, so it grows when the context length or the number of parallel request slots grows. - **PROCESSOR** — where that memory sits: in GPU memory (VRAM), in system RAM, or split across both. - **CONTEXT** — the context length, in tokens, that the model was actually loaded with. - **UNTIL** — when the model will be unloaded if no further request arrives (driven by `keep_alive`). ## How the PROCESSOR column is computed `/api/ps` reports two numbers per model: `size` (everything the loaded model uses) and `size_vram` (the part placed in GPU memory). The CLI turns them into text: | PROCESSOR reads | What it means | |---|---| | `100% GPU` | `size_vram` equals `size`: the whole loaded model is in VRAM | | `100% CPU` | `size_vram` is zero: nothing is on the GPU, the model runs from system RAM | | `48%/52% CPU/GPU` | 48% of the loaded size is in system RAM and 52% in VRAM | | `Unknown` | the two numbers are inconsistent (VRAM larger than the total) | The CPU share is simply `(size - size_vram) / size`, rounded. Two points trip people up: 1. **It is a memory placement, not a utilisation figure.** `100% CPU` does not mean the processor is pegged; it means no part of the model was placed on a GPU — because there is none Ollama can use, because the request asked for zero GPU layers, or because nothing fit. 2. **It describes the whole loaded model, context included.** A model whose weights would fit can still show a split once a large context or several parallel slots are added on top. ## Why a split is slow A model is a stack of layers, and every generated token passes through every layer in order. When Ollama cannot fit everything in VRAM, it places as many layers as fit on the GPU and leaves the rest in system memory, where the CPU computes them. The CPU-side layers are processed at system-memory speed, so they set the pace for every token: a model reading `100% GPU` at dozens of tokens per second can fall to a handful once a large share of it lands on the CPU. The reason decoding is bound by memory bandwidth belongs to the inference foundations; the operational point is that a split is a large slowdown, not a small one. ## What to do when you see a split Make the loaded model fit, then confirm with `ollama ps` that it reads `100% GPU`: 1. **Check the CONTEXT column first.** A context larger than the application needs is the most common cause; lower it with `num_ctx` or `OLLAMA_CONTEXT_LENGTH`. 2. **Check parallelism.** `OLLAMA_NUM_PARALLEL` multiplies the context the model is loaded with. 3. **Look for other loaded models.** Another row in `ollama ps` holds VRAM too; `ollama stop <model>` frees it. 4. **Consider the cache type or a smaller tag.** `OLLAMA_KV_CACHE_TYPE=q8_0` shrinks the context's memory; a smaller model shrinks the weights. `100% CPU` on a machine that has a GPU is a different problem: usually a driver or discovery issue, or a request that set `num_gpu` to 0, and the server log is where to look. ## Reading the rest of the row The **CONTEXT** column is the quickest way to learn what Ollama chose when nobody set a context, and the **UNTIL** column shows whether a model will stay warm (a time such as "4 minutes from now", or "Forever" when it was loaded with a negative `keep_alive`). Reading the whole row — size, placement, context and expiry — answers most "why is it slow" questions before any benchmarking starts.

  • How would a monitoring script detect a split without parsing the CLI table?
    Call `GET /api/ps` on the Ollama server. Each loaded model carries `size` and `size_vram`; when `size_vram` is less than `size`, part of the model is in system RAM. The CLI's percentage is just `(size - size_vram) / size`, so a script can alert on any model where the two differ.
  • Why is SIZE in `ollama ps` larger than the model file on disk?
    SIZE is the loaded runner's memory: the weights plus what is reserved for the context and its buffers. It therefore rises with the context length and with the number of parallel slots, and a model whose file would fit in VRAM can still show a CPU/GPU split once that context is added.

  • 100% CPU means the processor is fully utilised by the model
  • ollama ps lists every model pulled to the machine
  • A 48%/52% split only costs a few percent of speed
  • SIZE in ollama ps is the size of the model file on disk
  • The CPU/GPU split is fixed by the model and cannot be changed
open as a page

With Ollama, what is a Modelfile for, and how do you build and run a named model from one?

level: juniorimportance: must knowfreq 24%

basics

~20 s

A Modelfile bakes a base model plus fixed settings (PARAMETER values, a SYSTEM prompt, seeded MESSAGE turns, a TEMPLATE) into one named model. ollama create support-bot -f Modelfile builds it; any call by that name then uses it.

open as a page

In Ollama, what does `ollama pull llama3` fetch when you give no tag, and why pin a tag such as `llama3:70b` instead?

level: juniorimportance: must knowfreq 24%

basics

~10 s

A bare name expands to registry.ollama.ai/library/llama3:latest, and latest is only a tag its publisher can repoint. Pinning an explicit size-and-quant tag and comparing the ID column of ollama ls keeps machines on one build.

open as a page

With a local Ollama server, how do you repoint an OpenAI Python SDK app, and what must the client still supply?

level: juniorimportance: must knowfreq 24%

basics

~10 s

Set the client's base_url to http://localhost:11434/v1/, pass any placeholder api_key (the SDK requires one, Ollama ignores it), and name a model already pulled into Ollama, such as gpt-oss:20b.

open as a page

With Ollama 0.35.0, what context length does a model get when nothing sets one, and how do you change it?

level: middleimportance: should knowfreq 17%

basics

~20 s

Ollama 0.35.0 picks the default from total VRAM at startup: 4k tokens under 24 GiB, 32k from 24 to 48 GiB, 256k at 48 GiB and up. OLLAMA_CONTEXT_LENGTH changes it server-wide and options.num_ctx per request.

open as a page

An app on Ollama stalls for seconds on the first request after a quiet spell — which setting causes it, and how do you keep models loaded?

level: middleimportance: should knowfreq 12%

basics

~20 s

Ollama unloads an idle model after keep_alive, 5 minutes by default, so the next request reloads it. Send keep_alive with a longer duration or -1 per request, or set OLLAMA_KEEP_ALIVE on the server; 0 or ollama stop unloads.

open as a page

When an Ollama Modelfile builds FROM a library model, which settings does `ollama create` inherit, replace or merge from the base?

level: middleimportance: should knowfreq 9%

basics

~20 s

ollama create keeps the base's weights and every layer you omit. A new TEMPLATE, SYSTEM or MESSAGE set replaces the base's; PARAMETER merges key by key, so one stop line replaces the whole stop list; LICENSE appends.

open as a page

An OpenAI-SDK app calls an Ollama 0.35 model baked with SYSTEM, temperature 0.2 and num_ctx 16384 via /v1/chat/completions — which settings apply?

level: middleimportance: should knowfreq 12%

basics

~20 s

Only num_ctx is safe: /v1 cannot send a context size, so the Modelfile's 16384 applies. SYSTEM applies only when the client sends no system message first, and /v1 fills an omitted temperature with 1.0, overriding the baked 0.2.

open as a page

After a week of `ollama pull` fills a laptop's disk, how is Ollama's OLLAMA_MODELS store laid out, and how do you reclaim or move it?

level: middleimportance: should knowfreq 12%

basics

~10 s

OLLAMA_MODELS holds manifests/ (one small JSON per name:tag) and blobs/ (sha256-named layers shared between manifests). Free space with ollama rm, which deletes only blobs no other manifest uses; move the store by setting OLLAMA_MODELS.

open as a page

How do you pull a community GGUF from Hugging Face straight into Ollama, which quant arrives by default, and how does a private repo authenticate?

level: middleimportance: should knowfreq 9%

basics

~10 s

Run ollama pull hf.co/{username}/{repository}. Ollama takes Q4_K_M when the repo has it, else a reasonable quant present; add :{quantization} to choose. Private repos need Ollama's id_ed25519.pub added as a Hugging Face SSH key.

open as a page

In Ollama, what changes when an indexer moves from /api/embeddings to /api/embed, and what does /api/embed do with over-long inputs?

level: middleimportance: should knowfreq 8%

basics

~20 s

/api/embed takes input as a string or list and returns one L2-normalized vector per item under embeddings; /api/embeddings took one prompt and returned a raw vector. Over-long inputs are silently cut unless truncate is false.

open as a page

When does a client need Ollama's native /api/chat rather than /v1/chat/completions, and how do you read its streamed response?

level: middleimportance: should knowfreq 12%

basics

~20 s

Use /api/chat when a request needs Ollama's own controls directly: per-request options such as num_ctx, think levels, or a JSON schema in format. It streams by default as newline-delimited JSON; the last object has done: true and the timings.

open as a page

After raising OLLAMA_NUM_PARALLEL from 1 to 4 on a shared Ollama box, `ollama ps` shows a 14B model at 41%/59% CPU/GPU — why, and what do you change?

level: seniorimportance: should knowfreq 9%

basics

~10 s

Ollama loads a model with num_ctx times OLLAMA_NUM_PARALLEL tokens of context, so four slots roughly quadruple context memory and push layers off the GPU. Lower num_ctx, set OLLAMA_KV_CACHE_TYPE=q8_0, or reduce parallelism.

open as a page

A community GGUF imported into Ollama 0.35 with `FROM ./model.gguf` runs past its answer or prints role markers — how do you find and fix its prompt template?

level: seniorimportance: should knowfreq 7%

basics

~20 s

Check which template Ollama used: create may have matched the GGUF to a close-but-wrong built-in template, or the file has none. Write the model's format as a Go TEMPLATE with matching PARAMETER stop lines and re-run ollama create.

open as a page

On an Ollama 0.35 box meant to keep every prompt local, how can a model name still send prompts to ollama.com, and how do you forbid it?

level: seniorimportance: should knowfreq 7%

basics

~20 s

In Ollama 0.35.0 a tag ending in :cloud or -cloud runs on ollama.com: the local daemon forwards the request there once you are signed in. Set OLLAMA_NO_CLOUD=1 or disable_ollama_cloud in server.json and restart to forbid it.

open as a page

Against Ollama 0.35's /v1/chat/completions, an agent sending tool_choice 'required' and n=3 gets one choice and prose replies — why, and what else differs silently?

level: seniorimportance: should knowfreq 11%

basics

~10 s

Ollama 0.35's /v1/chat/completions does not support tool_choice, n, logit_bias or user and drops them without an error. It also sends temperature and top_p of 1.0 when omitted and has no context-size field.

open as a page

After upgrading to Ollama 0.35, an ADAPTER line and `ollama create -q q4_K_M` on a GGUF both fail — what changed, and what replaces each?

level: middleimportance: nice to knowfreq 4%

basics

~20 s

Since Ollama 0.34.1 ollama create rejects ADAPTER ("LoRA adapters are no longer supported"), so merge the adapter into the weights first. --quantize works only on safetensors imports (int4, int8, nvfp4, mxfp4, mxfp8); a GGUF must be quantized before import.

open as a page

Why does `ollama push mymodel` fail with "not authorized to push to this namespace", and how do you publish it for teammates?

level: middleimportance: nice to knowfreq 6%

basics

~10 s

An unqualified name lands in the library namespace, which you do not own. Add Ollama's public key to your ollama.com account, rename with ollama cp mymodel myuser/mymodel, push that name, and teammates pull myuser/mymodel.

open as a page

Pointing an Anthropic-SDK coding tool at a local Ollama 0.35 server, what do you configure, and which Messages API features stop working?

level: middleimportance: nice to knowfreq 5%

basics

~10 s

Set the base URL to http://localhost:11434 with a placeholder token and a local model name. Ollama's /v1/messages handles messages, streaming, tools and thinking, but not tool_choice, count_tokens, prompt caching, batches, PDFs or image URLs.

open as a page

With Ollama 0.35.0, what does OLLAMA_KV_CACHE_TYPE=q8_0 change, what must be enabled for it to work, and which models does it affect?

level: seniorimportance: nice to knowfreq 6%

basics

~20 s

OLLAMA_KV_CACHE_TYPE=q8_0 stores the KV cache at 8 bits, about half the memory of the default f16. It only works with flash attention enabled, which Ollama 0.35.0 does automatically where supported, and it applies to every model on the server.

open as a page