What actually happens when a process in a Docker container allocates past the container's `--memory` limit, and what does the `--memory-swap` flag change about that behavior?

## The mechanism `docker run --memory=512m` does one thing: it sets a memory limit on the control group (cgroup) that the container's processes live in — `memory.max` on cgroup v2, `memory.limit_in_bytes` on v1. A cgroup is the kernel's accounting and limiting unit for a set of processes; every page a process in that group touches is charged to it. A common misconception is that hitting the limit makes `malloc` fail. It usually does not. Linux overcommits: the allocator hands back an address range and only maps physical pages when they are first written. The reckoning therefore happens at page-fault time. When charging a new page would push the cgroup over its limit, the kernel first tries to reclaim — writing back and dropping page cache, evicting clean pages, swapping if swap is available to the cgroup. Only when reclaim cannot free enough does the **cgroup OOM killer** run. ## Cgroup OOM versus host OOM The cgroup OOM killer is scoped: it picks a victim from the processes in that cgroup, ranked by an OOM score roughly proportional to memory footprint, and sends SIGKILL. Nothing outside the container is touched — that is exactly the isolation you bought by setting a limit. The kernel logs a line such as `Memory cgroup out of memory: Killed process 1234 (java)`, visible through `dmesg` or the journal, and increments the `oom_kill` counter in the cgroup's `memory.events`. The **global** OOM killer is a different event: the whole host ran out, and the kernel picks a victim across all processes, which may be an unrelated container or a system daemon. Unlimited containers are the usual cause — the strongest argument for setting limits everywhere rather than nowhere. If the killed process is PID 1, the container's main process is gone, so the container stops. `docker inspect` then shows `State.ExitCode: 137` — 128 plus signal 9 — and `State.OOMKilled: true`. If the victim was a child (a forked worker, a shell), the container keeps running with a mysteriously missing subprocess, a far nastier failure to debug. SIGKILL cannot be caught, so there is no shutdown hook, no flush, no stack trace: the absence of application logs at the moment of death is itself the signal. ## Swap `--memory-swap` is the most misread flag in this area because it sets the **combined** memory-plus-swap ceiling, not the swap allowance. With `--memory=512m`: - `--memory-swap` omitted: total 1g, so 512m of swap is available. - `--memory-swap=512m` (equal to memory): swap effectively disabled; the container OOMs as soon as it exceeds 512m of RAM. - `--memory-swap=1g`: 512m RAM plus 512m swap. - `--memory-swap=-1`: unlimited swap, bounded only by host swap. Swap trades a kill for latency: the container survives but pages in and out, and throughput can collapse by orders of magnitude while everything still looks "up". For latency-sensitive services many teams deliberately disable swap so failure is fast and loud rather than slow and invisible. `--memory-swappiness` (0–100) tunes how eagerly the kernel swaps that container's anonymous pages. Swap accounting must be enabled in the kernel, or Docker warns and the swap settings are ignored. ## Related knobs - `--memory-reservation` is a **soft** limit: under host pressure the kernel tries to push the container back toward it, but it is not a kill boundary. Setting reservation below limit expresses "normally this much, occasionally more". - `--oom-kill-disable` stops the kernel killing anything in the cgroup; processes are frozen instead when the limit is hit. It is almost always a trap — a hung container that never recovers — and dangerous without a limit set. - `--oom-score-adj` biases which processes the *global* killer prefers. ## Practical takeaways Size the limit from measured peak, not average, plus headroom, because the limit is a hard cliff for an incompressible resource: unlike CPU, memory cannot simply be shared more slowly. Expect the OOM signature to be a silent restart loop with exit 137 and no application error, and confirm it from `docker inspect`, `memory.events` and kernel logs rather than guessing. And remember that everything the process touches counts toward the same cgroup limit — native buffers, thread stacks, page cache for files it writes — not just whatever the runtime reports as its own memory.