In llama.cpp 0.5.0, what do llama-server's -m/-hf, -ngl and -c flags control, and what happens when you leave them unset?
answer
- where the weights come from
- Hub repo plus an optional quant tag
- layers kept in VRAM, rest on CPU
- zero means the model's own length
- an on-by-default memory fitter
basics
~20 s-m loads a local GGUF, -hf downloads one from Hugging Face (Q4_K_M by default), -ngl caps layers in VRAM and -c sets context. Unset in llama.cpp 0.5.0, -ngl is auto, -c is the trained length, and --fit may shrink both.
solid answer
~40 s`-m path/to/model.gguf` loads a file you already have; `-hf <user>/<model>[:quant]` downloads one from Hugging Face into llama.cpp's cache, choosing Q4_K_M when no tag is given (or the repo's first file if there is no Q4_K_M), with `-hff` naming an exact file and the token read from `HF_TOKEN`. `-ngl` is the maximum number of layers stored in VRAM: a number, `auto` or `all`. Whatever does not fit runs on the CPU, the usual reason a server is slow. `-c` is the context size, and its default `0` means the length the model was trained with. In llama.cpp 0.5.0 both defaults pass through `--fit`, which is on by default: it shrinks the context, then the layer count, until each device keeps a free margin, and leaves alone any value you set yourself.
code
bash · 8 lines# local file, both values pinned: --fit will not change them
llama-server -m ./models/your-model-Q4_K_M.gguf -ngl all -c 16384
# from the Hub: Q4_K_M unless the tag names another quant
llama-server -hf ggml-org/gemma-3-4b-it-GGUF:Q4_K_M --port 8080
# same repo later, no network: use the cached copy only
llama-server -hf ggml-org/gemma-3-4b-it-GGUF:Q4_K_M --offlinego deeper
Recall the three flags and their defaults: -m or -hf for the weights, -ngl auto for layers in VRAM, -c 0 for the model's trained context.
Explain how -hf picks a file, why layers left on the CPU slow generation, and why a large default context costs memory before any request arrives.
Show you know --fit adjusts only what you left unset, and that pinning -ngl or -c changes what happens when memory runs short.
Argue when to rely on automatic fitting and when to pin values so every machine in a fleet starts the same way.
## What the three flags decide `llama-server` is the HTTP server that ships with llama.cpp. Before it can answer a single request it has to settle three things, and each one has its own flag: - **Which weights to load**: `-m` (`--model`) for a local file, or `-hf` (`--hf-repo`) to fetch one from Hugging Face. - **Where the layers run**: `-ngl` (`--n-gpu-layers`), the maximum number of the model's layers held in VRAM. The rest stay in system RAM and are computed on the CPU. - **How long the conversation can be**: `-c` (`--ctx-size`), the number of tokens of context the server allocates a KV cache for. Every one of these has a default in llama.cpp 0.5.0, and the defaults are smarter than they were in older guides. That is exactly why an interviewer asks about them: a candidate who copies `-ngl 99 -c 2048` from an old blog post gets a server that behaves differently from one started with no flags at all. ## Getting the weights: -m against -hf | Flag | What it takes | What happens | |---|---|---| | `-m` | a path to a `.gguf` file | loads the file as is; nothing is downloaded | | `-hf` | `<user>/<model>[:quant]` | downloads from the Hub into the cache; the quant tag is case-insensitive and defaults to `Q4_K_M`, falling back to the repo's first file if there is no `Q4_K_M` | | `-hff` | a file name inside the repo | overrides the quant tag and picks that exact file | | `-hft` | an access token | defaults to the `HF_TOKEN` environment variable, for gated or private repos | `-hf` also fetches a multimodal projector when the repo has one (turn that off with `--no-mmproj`). Downloads go into llama.cpp's cache, whose location the `LLAMA_CACHE` environment variable controls; `-cl` (`--cache-list`) lists what is cached, and `--offline` forces the server to use the cache and never touch the network. What a label such as `Q4_K_M` means is the quantization material's subject; here it is simply a name the flag matches. ## -ngl: how much of the model lives on the GPU `-ngl` accepts an exact number, `auto` or `all`, and the default is **`auto`**. In practice `auto` starts from every layer on the GPU and lets `--fit` back off if memory runs short. A number is a ceiling: `-ngl 20` keeps at most 20 layers in VRAM and computes the rest on the CPU, which is far slower per token. Two consequences matter: 1. Typing any explicit value, `-ngl 99` and `-ngl all` included, counts as setting it yourself, so `--fit` will not lower it. If the model and its cache then do not fit, you find out at load time. 2. Leaving it unset is usually right on a single GPU, because the server already tries to put everything there. ## -c: the context window, and the default that surprises people `-c` defaults to **`0`, meaning the context the model was trained with**, read from the GGUF's metadata. That can be very large, and the KV cache that backs it costs memory in proportion to its length (why that is so is the KV-cache material's subject). The README's quick-start line uses `-c 2048`, but that is an example, not the default. The server also splits its context into **slots** for concurrent requests; `-np` defaults to auto, which in 0.5.0 means four slots sharing one unified KV buffer, so the context you choose is a pool, not one person's window. ## What --fit does with what you left unset `--fit` is **on by default** in llama.cpp 0.5.0. Before loading, it estimates the memory the model, cache and compute buffers need on each device and, if they do not fit with a margin free (`-fitt`, default 1024 MiB per device), it adjusts **only the arguments you did not set**: - first the context, if `-c` was left out, but never below the floor `-fitc` (default 4096); - then the GPU layer count, if `-ngl` was left at `auto`. So a bare `llama-server -m model.gguf` on a small GPU may start with a shorter context than the model supports, or with some layers on the CPU, and say so in its log. Pinning `-c` and `-ngl` yourself, or passing `--fit off`, takes the decision back. ## Checking what you actually got 1. Read the startup log: the line `initializing, n_slots = ..., n_ctx_slot = ...` shows the slot count and each slot's context. 2. Query `GET /slots` for each slot's `n_ctx`, or `GET /props` for the server's defaults and the model's chat template. 3. Watch tokens per second in a response's `timings` object; a sudden drop after a config change usually means layers moved to the CPU. 4. Run with `-v` to see the fitting messages, such as how far the context was reduced. The server listens on `127.0.0.1:8080` by default, and the built-in web UI is served at the same address, so the first test is simply opening that page.
- How do you run llama-server on a machine without internet after using -hf?Downloads land in llama.cpp's cache, which the `LLAMA_CACHE` environment variable relocates, and `-cl` lists its contents. Restart with the same `-hf` argument plus `--offline`, which forces the cache and blocks network access, or copy the GGUF across and load it with `-m`. A multimodal repo's projector arrives with `-hf` automatically, so copy that file too and pass it with `-mm`.
- Why can `-ngl 99` copied from an older guide behave differently in llama.cpp 0.5.0?`-ngl` already defaults to `auto`, which starts with every layer on the GPU and lets `--fit` back off when memory is short. Any explicit number, 99 included, counts as your choice, so `--fit` stops lowering the layer count: it may still shrink an unset context, but if the model will not fit you find out at load time. Leave `-ngl` unset unless you mean to pin it.
- llama-server offloads nothing to the GPU unless you pass -ngl.
- Leaving -c unset gives llama-server a 2048-token context.
- -hf without a quant tag downloads the full-precision file.
- -ngl sets how many gigabytes of VRAM llama-server may use.
- --fit overrides the -c and -ngl values you typed yourself.
Assumes llama.cpp 0.5.0, where -ngl defaults to auto, -c 0 means the trained context and --fit is on by default.