What does vLLM's --api-key flag actually protect on the server?

## What the flag does, precisely Starting the server with `--api-key <token>` (equivalently, setting `VLLM_API_KEY`) adds an HTTP middleware. For requests to the API routes, it reads the `Authorization` header, expects the literal form `Bearer <token>`, compares it against the configured value, and returns 401 if it does not match. That is the whole mechanism, and its simplicity is the point: it exists so that an OpenAI-compatible client, which already sends its key in exactly that header, works unchanged against a server that wants one. Because the SDKs always send *something* in that header, an unauthenticated vLLM ignores whatever placeholder arrives. Turning the flag on turns that placeholder into a real requirement without changing a line of client code. ## Everything it does not do **No identity.** The token authenticates a secret, not a caller. Two teams sharing one server share one token, so you cannot attribute usage, cannot revoke one consumer without breaking the other, and cannot grant one team a smaller allowance. Any per-tenant story you want has to live somewhere else. **No quotas or rate limits.** vLLM will accept whatever arrives and queue it. A misbehaving client with a valid token can saturate the batch, and the visible symptom for everyone else is climbing time-to-first-token rather than a rejection. **No request-size policy.** An authorized caller can request generations up to the server's configured context length. A handful of maximum-length requests occupy KV cache that dozens of short requests could have used, so "authorized" is nowhere near "harmless". **No rotation.** Changing the token is a process restart, which on a large model means minutes of downtime while weights reload. Rotation therefore needs a fronting layer that can swap credentials without touching the GPU process. **No TLS.** The flag is about a header, not about transport. Sent over plain HTTP, the token is on the wire in clear text on every request. ## The routes that stay open The key check covers the API paths. Operational routes — the health endpoint used by orchestrator probes and the Prometheus metrics endpoint — sit outside that prefix and answer without a token. This is convenient (your probe and your scraper do not need the secret) and it is a leak: metrics disclose which model is loaded, how much traffic it takes, queue depth and latency distributions. Anyone who can reach the port can profile your deployment even without the key. The correct conclusion is not "vLLM should protect them" but "the port should not be reachable by anyone you would not tell". ## What production actually looks like Bind the server to a private interface or a cluster-internal service, never a public address, and enforce that with network policy or security groups rather than trusting a configuration flag. Put a gateway, ingress or reverse proxy in front, and give it the jobs vLLM does not do: terminate TLS, authenticate real callers (per-tenant keys, mTLS, OIDC — whatever your organization already uses), enforce per-tenant rate limits and concurrency caps, cap request body size and requested output length, and emit usage records for chargeback. Keep `--api-key` on as well, as defence in depth: it means a pod that is accidentally exposed, or a misconfigured route that bypasses the gateway, still refuses anonymous traffic. One more habit worth adopting: pass the token through the environment rather than the command line. Process arguments are readable from the process table and land in orchestrator manifests, shell history and crash dumps, so `VLLM_API_KEY` sourced from a secret store is strictly better than an inline `--api-key` in a Kubernetes command array. ## How to answer this in an interview The question is really testing whether you treat an inference server as an application server. A model endpoint is a compute-expensive, unusually easy-to-abuse resource: every request costs GPU seconds, and the cost per request varies by three orders of magnitude depending on prompt and output length. A single shared bearer token is the least interesting part of protecting that. Say plainly what the flag is (a tripwire), name the missing controls, and put them at the layer that already owns them in your stack.