Container OOM Kills

## Two different failures that people treat as one `MemoryError` is raised *inside* your process: some allocation returned NULL, CPython noticed, and raised an exception at that exact line. The stack unwinds, `except` blocks run, `finally` blocks run, and the process is still alive. A **container OOM kill** happens *to* your process: the kernel decides, sends SIGKILL, and the interpreter is stopped between two bytecodes with no exception, no cleanup and no traceback. Which one you get is decided by *where the memory pressure becomes visible*, and that is a kernel question, not a Python question. ## Why the kill usually wins On Linux with the default **heuristic overcommit** policy, a large allocation request is mostly bookkeeping: the kernel reserves address space and returns success without committing physical memory. Pages are charged only on **first touch**, when the fault handler backs the page. A cgroup memory limit is enforced at exactly that charge point. So the sequence in a container is: 1. your allocation *succeeds*, 2. you begin filling the object, 3. and somewhere in the middle of an ordinary write the charge cannot be met, 4. reclaim fails, 5. and the cgroup OOM killer fires. There was never a NULL for CPython to turn into an exception, which is why wrapping the code in `try/except MemoryError` changes nothing at all. Worse, the kernel picks a victim inside the cgroup by score, so in a multi-process container the process that dies may not be the one that allocated -- a small supervisor can be killed for a large worker's growth. ## When you genuinely do get MemoryError Several situations still produce the catchable form. - A single request the kernel refuses on sight -- ask for a few exabytes and the heuristic says no immediately. - A per-process address-space ceiling configured for the process, which makes the allocator itself fail rather than the kernel act later. - A host with overcommit disabled entirely, where every mapping must be backed by memory plus swap at request time and `malloc` really can return NULL. - And a handful of cases CPython checks itself, such as a computed length that cannot possibly be allocated. There is also a real distinction between failing to allocate an object's payload and failing to allocate the small internal structure: both surface as `MemoryError`, but only the first is proportional to your data. ## Catching MemoryError is not a recovery strategy Even when the exception does arrive, the handler runs in a process that has just proved it cannot get memory. Formatting a log message, building a traceback, or constructing the exception's own message may need allocations that fail again. Other threads are hitting the same wall concurrently, so state can be inconsistent in ways your handler does not expect. The only handlers that reliably work are ones that **free something already reserved** -- dropping a cache you deliberately kept droppable -- and then either retry or exit deliberately. Treating `except MemoryError: continue` as backpressure produces a process that limps along corrupting throughput instead of restarting clean. ## Making failure catchable on purpose If you want a degradation path rather than a sudden kill, you have to move the ceiling *inside* the process, below the platform's limit, so your own code notices first. Concretely: - bound how much you buffer per work item, - cap the number of items in flight, - reject or spool payloads above a size you chose, - and stream rather than reading whole objects into memory. Those checks raise exceptions you wrote, at points you control, with headroom left to handle them. A process-level address-space ceiling can play the same role at a coarser grain, turning a would-be kill into a `MemoryError` at the allocation site -- at the cost that the exception can arrive anywhere, including inside library code that was never written to survive it. ## The interview point The distinction is diagnostic, not academic. - A traceback ending in `MemoryError` tells you a single allocation was too big for what remained and the process kept running -- look at that allocation. - An exit status of 137 with no traceback tells you the kernel enforced a limit against your resident footprint -- look at the peak, at concurrency, and at how much memory the process is holding that it never returns.