In the cgroup v2 memory controller on Linux, what is the difference between writing a limit to memory.high and writing one to memory.max?

## The knobs, and what each one is for The cgroup v2 memory controller exposes a small set of files in every cgroup that has `memory` enabled by its parent's `cgroup.subtree_control`: - `memory.current` — bytes currently charged to this cgroup and its descendants. - `memory.min` — hard protection; memory below this is never reclaimed from the cgroup, even under global pressure. - `memory.low` — best-effort protection; reclaim avoids this cgroup while it is under the value, unless there is no alternative. - `memory.high` — throttle limit. - `memory.max` — hard limit. - `memory.swap.max` — cap on swap usage for the cgroup. - `memory.events` — a counter file with `low`, `high`, `max`, `oom` and `oom_kill` fields. - `memory.stat` — a detailed breakdown (anon, file, slab, and much more). The four limits form a ladder: `min` and `low` protect memory *for* the cgroup, `high` and `max` protect the rest of the system *from* it. ## memory.high: throttle, do not kill When `memory.current` exceeds `memory.high`, the kernel does two things. First, it runs direct reclaim against that cgroup to try to bring usage back down — writing back dirty page cache, dropping clean pages, swapping anonymous memory if swap is available and permitted. Second, and this is the part people miss, it *penalises the allocating task*: on return to userspace after an allocation that pushed the cgroup over, the task is put to sleep, and the sleep grows with how far over the limit the cgroup is. The result is a workload that keeps running but gets slower. That is deliberate — it converts a hard failure into back-pressure, giving an operator or a supervisor time to react. Because there is no enforcement wall, `memory.current` can and does sit above `memory.high` indefinitely if the workload keeps allocating faster than reclaim frees. The cost is that the throttling is invisible in ordinary CPU metrics: the process is not consuming CPU, it is sleeping and thrashing. This is what `memory.pressure` (the PSI file) is for — it reports the share of wall time tasks in the cgroup spent stalled waiting on memory. ## memory.max: the wall `memory.max` is enforced at charge time. When a page charge would push the cgroup over the limit, the kernel first tries reclaim within the cgroup. If reclaim cannot free enough, the cgroup out-of-memory killer runs and kills a process **inside that cgroup** — the rest of the host is untouched. The killed process dies from `SIGKILL`, which it cannot catch, so it writes nothing to its own log about why it went away; the evidence is in `memory.events` (`oom_kill` increments) and in the kernel log. Two related knobs matter here. `memory.oom.group` makes the kill atomic for the whole cgroup — either every process in it dies or none does, which is what you want when the processes are one logical unit and a survivor is useless. And `memory.swap.max` decides whether anonymous memory can be pushed to swap before the limit is hit; setting it to `0` means anonymous pressure goes straight to OOM. ```bash cat /sys/fs/cgroup/svc/memory.events # low 0 # high 41 # max 3 # oom 1 # oom_kill 1 ``` Reading that file is the fastest way to answer "was this cgroup throttled, squeezed, or killed?" — `high` counts times usage went over the throttle, `max` counts times an allocation was blocked at the hard limit, and `oom_kill` counts processes actually killed. ## Choosing values The useful configuration is both, with a gap: `memory.high` at the level where you want to be told something is wrong, `memory.max` a margin above it as the backstop that protects the host. A workload with a genuine leak will cross `high`, stall visibly in `memory.pressure`, and only die at `max` — by which time you have signal. Setting only `max` gives you a binary outcome: fine, then dead. Setting only `high` protects nothing, because a runaway allocator can outrun reclaim and drag the whole machine into pressure. One caveat worth stating in an interview: `memory.high` throttling is not free. A workload parked just above `high` burns real time in reclaim, and if that memory is page cache it needs, the cgroup can thrash — repeatedly evicting and re-reading the same pages. If `memory.pressure` shows sustained stall, the correct fix is more memory or less work, not a higher throttle. ## Defaults Both files default to the literal string `max`, meaning no limit, and both accept `max` to reset. Values are written in bytes, with the usual `K`/`M`/`G` suffixes accepted. Limits are hierarchical: a child cannot escape an ancestor's `memory.max`, so the effective limit is the smallest along the path from the root.