skip to content

Before sharing one llama-server with a team over the LAN, which flags and default routes do you change, and why?

level: middleimportance: should knowfreq 8%

answer

  1. loopback-only until you say otherwise
  2. no key means no authentication
  3. Bearer or the Anthropic-style header
  4. health stays public, slots do not
  5. CORS reflects any origin by default

basics

~10 s

Bind it with --host, since the default is 127.0.0.1, and add --api-key or --api-key-file, because without one there is no authentication. Then narrow --cors-origins, decide on /slots and /metrics, and never enable --tools.

solid answer

~40 s

llama-server listens on `127.0.0.1:8080`, so other machines cannot reach it until `--host` names a LAN address or `0.0.0.0`. With no `--api-key` there is **no authentication** at all. `--api-key` takes a comma-separated list and `--api-key-file` one key per line; clients send `Authorization: Bearer <key>` or `X-Api-Key`, and every route except `/health`, `/v1/health` and the web UI's files then returns 401 without one. Check the defaults: `/slots` is on (`--no-slots` hides it), `/metrics` is off until `--metrics`, and `POST /props` needs `--props`. CORS reflects any origin with credentials by default, so set `--cors-origins` to your frontend. TLS needs `--ssl-cert-file` and `--ssl-key-file` on an OpenSSL build, or a reverse proxy. Leave `--tools` and `--agent` off: they let the model touch the host's files and shell.

code

bash · 13 lines
bash
# keys file: one key per line, '#' starts a comment
printf '# team keys\nteam-key-1-placeholder\nteam-key-2-placeholder\n' > /etc/llama/keys.txt
chmod 600 /etc/llama/keys.txt

llama-server -m ./models/your-model-Q4_K_M.gguf \
  --host 192.168.1.20 --port 8080 \
  --api-key-file /etc/llama/keys.txt \
  --cors-origins http://192.168.1.20:3000 \
  --metrics

# /health stays public; every other route needs a key
curl -s http://192.168.1.20:8080/health
curl -s http://192.168.1.20:8080/slots -H "Authorization: Bearer team-key-1-placeholder"

go deeper

for a junior

Recall that llama-server binds 127.0.0.1:8080 and has no authentication until you pass --api-key or --api-key-file.

for a middle

Explain which routes stay public with a key set, which monitoring routes are on or off by default, and how clients send the key.

for a senior

Harden a team box: bind one interface, key it, narrow CORS, add TLS or a proxy, and keep --tools, --agent and --props off.

for a principal

Decide when a single keyed server is enough and when per-user identity, quotas and audit logs mean a proxy in front is required.

## Out of the box it is a private server A freshly started llama-server is built for one person on one machine. It binds `127.0.0.1` on port `8080`, so nothing else on the network can connect, and it has **no authentication**: anything that can reach the port can use every route. Both are reasonable defaults for a laptop and both are wrong for a shared box, so opening it to a team is a short, deliberate checklist rather than a single flag. ## Step one: bind and authenticate together Changing the bind without adding a key publishes an open model server to the whole network, so do the two in one change: 1. **Bind.** `--host` takes one or more comma-separated addresses, or a UNIX socket path ending in `.sock`. Prefer the LAN interface's own address to `0.0.0.0`, which listens on every interface, VPNs and containers included. 2. **Authenticate.** `--api-key` takes one key or a comma-separated list; `--api-key-file` reads one key per line and treats lines starting with `#` as comments. The keys are read once at startup, so a changed file takes effect on restart. 3. **Send the key.** Clients put it in `Authorization: Bearer <key>`. If that header is missing, the server also checks `X-Api-Key`, which is what Anthropic-style clients of `/v1/messages` send. A wrong or missing key gets a 401 with `Invalid API Key`. Every key is equal. The server checks membership in the list and nothing else: there is no per-key identity, quota or usage report, so give each person their own key only so you can revoke one, and put real accounting in a proxy if you need it. ## Which routes stay open, and which defaults to revisit | Route or feature | Default | With `--api-key` set | |---|---|---| | `GET /health`, `GET /v1/health` | on | still public | | web UI files | on (`--no-ui` turns it off) | page loads without a key; its API calls need one | | `GET /slots` | on (`--no-slots` turns it off) | requires the key | | `GET /metrics` | **off** until `--metrics` | requires the key | | `POST /props` | off until `--props` | requires the key | `/slots` reports every slot's sampling settings and state, which is handy for operators and more than an ordinary user needs; `--no-slots` hides it. `/metrics` is a Prometheus exporter, so the scraper needs a key too. ## Browsers, CORS and the web UI By default the server **reflects any `Origin` header back with credentials allowed**. That is harmless for a read-only local tool, but on a LAN it means any web page a teammate opens could call the server from their browser. Set `--cors-origins` to your frontend's origin, or to the special value `localhost`. The README's own guidance is: on a local network, set `--cors-origins` to the frontend's origin; in public, set an API key and put the server behind a reverse proxy. ## Mistakes that open more than intended - **Publishing the Docker example as is.** The README's container command passes `--host 0.0.0.0` so the port mapping works; on a host network that means every interface, with no key unless you add one. - **Sharing one key with everyone.** Revoking it locks the whole team out; a key per person in `--api-key-file` lets you remove one line and restart. - **Trusting the web UI to be private.** Its page is served without a key, so the server's address alone tells anyone on the LAN that a model is running there. - **Leaving `/slots` on for users who do not need it.** It is harmless to operators and needless detail for everyone else. ## Encryption and the features to leave off - **TLS**: `--ssl-cert-file` and `--ssl-key-file` work only on a build made with `-DLLAMA_OPENSSL=ON`; otherwise terminate TLS in a reverse proxy, which is also where per-user logging belongs. Without TLS, keys cross the LAN in clear text. - **`--tools` and `--agent`** are experimental switches that give the model built-in tools to read and write files and run shell commands on the host, reachable through the API and the web UI. The README says not to enable them in untrusted environments. They narrow `--cors-origins` to `localhost` by default, but anyone holding a key can still drive them. On a shared server, leave them off. - **`--props`** lets a client change global server properties; keep it off unless you need it.

  • What does llama-server's --metrics expose, and who can read it once keys are set?
    It enables `GET /metrics`, a Prometheus exporter with counters such as `llamacpp:prompt_tokens_total` and gauges such as `llamacpp:requests_processing` and `llamacpp:requests_deferred`. It is off by default. Once on, it sits behind `--api-key` like every route except `/health` and `/v1/health`, so the scraper needs a key too. In router mode the query needs `?model=`.
  • Why are llama-server's --tools and --agent dangerous on a shared box?
    They are experimental switches that let the model read and write files and run shell commands on the host through the API and the web UI, and the README says not to enable them in untrusted environments. They narrow `--cors-origins` to `localhost` by default, but any key holder can still drive them. If you need them, `--tools-runtime` can run the tools inside a container instead of on the host.

  • llama-server listens on every network interface by default.
  • Without --api-key, llama-server rejects requests that carry no key.
  • --api-key also locks the /health route.
  • /slots stays hidden unless you start llama-server with --slots.
  • Each llama-server API key carries its own quota and user identity.

Assumes llama.cpp 0.5.0: --host 127.0.0.1, /slots on, /metrics off, and CORS reflecting any origin unless --cors-origins is set.