skip to content

In llama.cpp 0.5.0, an old launch script's --mlock and --no-mmap flags are rejected; what replaced them, and what does each --load-mode value do?

level: middleimportance: nice to knowfreq 4%

answer

  1. one flag where there were two
  2. -lm, default auto
  3. mmap maps, mlock pins
  4. mmap+mlock and dio
  5. LLAMA_ARG_LOAD_MODE in a service file

basics

~20 s

llama.cpp 0.5.0 replaced both with -lm / --load-mode: auto (the default, mmap unless a device cannot), none, mmap, mlock, mmap+mlock or dio. --no-mmap becomes none; --mlock becomes mmap+mlock, or mlock when mapping was off too.

solid answer

~40 s

In llama.cpp 0.5.0 `--mlock` and `--no-mmap` no longer exist; one option, `-lm` / `--load-mode` (env `LLAMA_ARG_LOAD_MODE`), says how the GGUF gets into memory. `auto` is the default and memory-maps the file unless a device does not support it. `mmap` maps the file so pages load on demand and the operating system may evict them. `none` reads the model without mapping, which is what `--no-mmap` used to mean. `mlock` forces the model to stay in RAM rather than being swapped or compressed, and `mmap+mlock` does both. `dio` uses direct I/O where available. So an old `--mlock` becomes `--load-mode mlock` or `mmap+mlock`. The bindings expose the same choice differently: llama-cpp-python 0.3.36 keeps `use_mmap` and `use_mlock` keyword arguments and maps them onto the load mode.

code

bash · 11 lines
bash
# Before (older llama.cpp):
#   llama-server -m ./models/your-model.gguf --mlock

# llama.cpp 0.5.0: map the file and pin it in RAM
llama-server -m ./models/your-model.gguf --load-mode mmap+mlock

# Same setting from a service environment
LLAMA_ARG_LOAD_MODE=mmap+mlock llama-server -m ./models/your-model.gguf

# Compare load modes in one benchmark run
llama-bench -m ./models/your-model.gguf -lm mmap,none

go deeper

for a junior

Recall that llama.cpp 0.5.0 uses --load-mode instead of --mlock and --no-mmap, and that the default, auto, memory-maps the model file.

for a middle

Explain what mapping and locking each do to start-up time, page eviction and RAM use, and translate an old flag pair into the right --load-mode value.

for a senior

Diagnose a first-request stall after idle as evicted mapped pages, choose between mmap+mlock and none on a shared host, and know how each binding spells the same setting.

for a principal

Set a load-mode policy for hosts that share RAM with other services, trading start-up speed and page-cache sharing against guaranteed residency and the memory it denies to neighbours.

## Two flags became one option Older llama.cpp guides start the tools with `--mlock` (pin the model in RAM) or `--no-mmap` (do not memory-map the file). In llama.cpp 0.5.0 both flags are gone from the argument parser, so a launch script copied from such a guide fails at start-up. Their job moved to a single option: - **`-lm` / `--load-mode MODE`**, default `auto`. - The same setting from the environment: **`LLAMA_ARG_LOAD_MODE`**, handy in a systemd unit or a container. - `llama-bench` accepts `-lm` too, with a comma-separated list, so you can benchmark several modes in one run. In the C API the same choice is the `load_mode` field of `llama_model_params`, an `enum llama_load_mode` in `include/llama.h`. ## What each mode does | Value | What llama.cpp does | Replaces | |---|---|---| | `auto` | memory-maps the file unless a device does not support it | the old default | | `none` | no special loading mode: reads tensors without mapping | `--no-mmap` | | `mmap` | memory-maps the model file | the old default, made explicit | | `mlock` | forces the system to keep the model in RAM rather than swapping or compressing it | `--mlock --no-mmap` | | `mmap+mlock` | memory-maps the file and locks it in RAM | `--mlock` | | `dio` | reads the file with direct I/O, where available | (new) | ## Choosing one **Memory-mapping** (`mmap`, and `auto` in most cases) is the usual choice: - Start-up is fast, because the file is mapped rather than copied; pages arrive from disk as the model touches them. - The mapped pages sit in the operating system's page cache, so a second process mapping the same file shares them, and a restart finds them still cached. - The catch is that the OS may **evict** mapped pages under memory pressure. On a RAM-tight machine running other work, the first request after a quiet spell can stall while pages are read back from disk. **Locking** (`mlock`, `mmap+mlock`) is the cure for that stall: - The model stays resident instead of being swapped out or compressed. - It needs the RAM to be genuinely free, and the operating system has to let the process lock that much memory; on a machine that cannot spare it, locking a large model pushes everything else out instead. **Not mapping** (`none`) copies the weights into ordinary memory. The tool's own help notes that loading is slower without mmap but can reduce pageouts when you are not locking. **Direct I/O** (`dio`) reads the file while bypassing the page cache, where the platform supports it. It suits a box where filling the cache with one huge file would push out data other services need. When every layer is offloaded to a GPU, the load mode mostly changes start-up time and host memory use, not generation speed; it matters most for models that run partly or wholly from system RAM. ## The same choice in the bindings The bindings did not all rename their options: 1. **llama-cpp-python 0.3.36** keeps `use_mmap=True` and `use_mlock=False` as `Llama(...)` keyword arguments and converts them: both true gives `mmap+mlock`, only `use_mlock` gives `mlock`, only `use_mmap` gives `mmap`, neither gives `none`. It never selects `auto`, and passing a `lora_path` switches mmap off. 2. **node-llama-cpp 3.22.1** has `useMmap` (default `"auto"`), `useMlock` (default `false`) and `useDirectIo` on `loadModel()`. 3. Code written directly against `llama.h` sets `params.load_mode`, and can ask `llama_supports_mmap()` and `llama_supports_mlock()` first. ## Symptoms that point at the load mode - **The first request after a quiet period is slow, later ones are fine.** Mapped pages were evicted and are being read back from disk; locking, or freeing memory elsewhere, cures it. - **The process looks much smaller than the model file.** That is mapping working as designed: the weights are file-backed pages in the page cache, not private memory. - **The whole machine slows down after a lock.** A locked model cannot give memory back, so other programs are squeezed instead; node-llama-cpp's own documentation for `useMlock` warns that forcing the model into memory can crash a system that lacks the resources. - **Load takes noticeably longer after switching to `none`.** Expected: the file is read up front rather than mapped. ## Migrating a script 1. Replace `--no-mmap` with `--load-mode none`. 2. Replace `--mlock` with `--load-mode mmap+mlock`, or `mlock` if the script also had `--no-mmap`. 3. If the script set nothing, leave it alone: `auto` already maps the file. 4. Re-check memory on the host after the change, because a locked model no longer gives RAM back under pressure.

  • With mmap, the llama.cpp process looks far smaller than the model file. Is the model not loaded?
    It is mapped, not copied: the weights are file-backed pages in the operating system's page cache, counted against the file rather than as the process's private memory. Pages arrive as they are touched and can be evicted under pressure, which is why a stall after idle points at eviction, and `--load-mode mmap+mlock` is the fix.
  • In llama-cpp-python 0.3.36, how do you get the equivalent of --load-mode mlock?
    Pass `use_mlock=True, use_mmap=False` to `Llama(...)`; the constructor maps that pair to the `mlock` load mode. Keeping the default `use_mmap=True` with `use_mlock=True` gives `mmap+mlock` instead. The binding has no `load_mode` keyword of its own and never chooses `auto`.

Memory-mapping a model is like reading a reference book from the library shelf: pages come to you as you need them, and the librarian may reshelve them when the room fills up, so the next look means another walk. Locking it is checking the book out: it stays on your desk, and that desk space is no longer anyone else's.

  • Keeps passing --mlock to llama.cpp 0.5.0 tools and expects it to work.
  • Thinks --load-mode none memory-maps the file.
  • Believes mmap reads the whole model into private memory at start-up.
  • Says mlock costs nothing on a machine short of RAM.
  • Assumes llama-cpp-python exposes a load_mode keyword like the CLI.

Assumes llama.cpp 0.5.0, whose tools take -lm / --load-mode; --mlock and --no-mmap no longer exist, and llama-cpp-python 0.3.36 still uses use_mmap and use_mlock.