What does the PYTHONMALLOC environment variable control in CPython?
answer
- Read before any Python code runs
- Which allocator, plus optional hooks
- Lets external C tooling see each object
- Guard bytes and fill patterns
- A diagnostic switch, never a tuning knob
basics
~20 sPYTHONMALLOC picks which allocator CPython installs and whether debug hooks wrap it: pymalloc by default, malloc to route every request to the C library, and the debug variants to add guard bytes and fill patterns.
solid answer
~40 sIt is read during interpreter pre-initialisation, before any Python code runs. `default` and `pymalloc` keep the small-object allocator; `malloc` sends every request, including the sub-512-byte ones, to the C library; `debug`, `pymalloc_debug` and `malloc_debug` add CPython's debug hooks, which fill fresh memory with a pattern, put guard bytes around each block to catch overruns and underruns, poison freed memory to expose use-after-free, and check that a block is released by the same allocator domain that produced it. `-X dev` enables those hooks without the variable. You set `malloc` so an external C memory checker sees one allocation per object instead of one 1 MiB arena. Side effects matter: under `malloc`, `sys.getallocatedblocks()` reports 0 and `sys._debugmallocstats()` loses its pool and arena sections. An unknown value is a fatal pre-initialisation error.
code
console · 1 linePYTHONMALLOC=malloc python3 -c "import sys; print(sys.getallocatedblocks())"go deeper
It is enough to know CPython's allocator is selectable through an environment variable and that the default is the right choice; you would not set it during ordinary application work.
Be able to name the main values and what each does, and explain why routing everything to the system allocator is what makes external C memory tooling see individual objects.
Show when you would actually reach for it: chasing memory corruption from a C extension or an embedding, using the debug hooks' guard bytes and fill patterns, and knowing that measurements taken under it do not transfer to the default build.
Judge it as a diagnostic capability with a cost: who is allowed to flip it, in which environment, and how you keep the resulting memory numbers from being quoted as if they described production.
### What the variable selects `PYTHONMALLOC` is read during interpreter pre-initialisation, before any Python code runs, and it chooses which allocator CPython installs and whether the debug hooks are wrapped around it. The values on CPython 3.14's standard build: * `default` — the built-in choice: pymalloc for objects, the system allocator for raw memory. * `pymalloc` — the small-object allocator explicitly. * `malloc` — route **everything** to the C library's `malloc`, including the sub-512-byte requests pymalloc would normally serve. * `pymalloc_debug`, `malloc_debug` — the same two, with debug hooks installed. * `debug` — the default allocators plus debug hooks. The free-threaded build (experimental in 3.13, officially supported in 3.14) uses **mimalloc** as its object allocator and accepts the corresponding mimalloc values there. An unrecognised value is fatal: the interpreter aborts during pre-initialisation with "PYTHONMALLOC: unknown allocator" rather than falling back silently. ### Why anyone sets it **`PYTHONMALLOC=malloc` makes external C memory tooling usable.** A native leak detector or bounds checker instruments `malloc`/`free`. With pymalloc in the way, thousands of Python objects share one 1 MiB arena that the tool sees as a single allocation, so an overrun from one object into its neighbour is invisible and every real allocation site is hidden behind the arena. Routing every request through `malloc` gives the tool one allocation per object, at a large speed cost — this is a debugging configuration, never a production one. **The debug hooks catch C-extension memory bugs from inside CPython.** They fill freshly allocated memory with a recognisable byte pattern (so use-before-initialise shows up), write guard bytes before and after each block to detect buffer overruns and underruns, fill freed memory with a different pattern to expose use-after-free, and check that memory allocated in one allocator domain is released in the same one. They abort with a diagnostic at the point of detection instead of leaving corruption to surface somewhere unrelated hours later. `-X dev` (development mode) turns the same hooks on without setting the variable, and `PYTHONDEVMODE=1` is its environment equivalent. ### The companion variable `PYTHONMALLOCSTATS=1` makes CPython dump the pymalloc statistics — the size-class table plus the arena summary — every time a new arena is created and again at shutdown. It is the non-interactive sibling of calling `sys._debugmallocstats()` yourself, and it only has anything to say while pymalloc is actually the allocator in use. ```console PYTHONMALLOC=malloc python3 -c "import sys; print(sys.getallocatedblocks())" ``` ### The observable side effects worth knowing Switching away from pymalloc changes what the introspection tools report, which surprises people mid-investigation: * `sys.getallocatedblocks()` returns **0** under `PYTHONMALLOC=malloc` — it counts pymalloc blocks, and there are none. * `sys._debugmallocstats()` still runs but prints only the object free-list section; the size-class table and arena counts are gone, because there are no pools or arenas. * RSS behaviour changes too: the arena-occupancy effect disappears and retention becomes entirely the system allocator's business, so a memory profile measured under `malloc` does not predict production behaviour under the default build. ### How to talk about it The useful framing is that `PYTHONMALLOC` is a **diagnostic switch, not a tuning knob**. There is no production workload where turning pymalloc off is the answer to a performance or memory problem — small-object allocation gets slower and the fragmentation story does not improve, it just moves to another allocator. You reach for it when a C extension or embedding is corrupting memory and you need either CPython's own guard bytes or an external checker's view, and you turn it off again as soon as the bug is found.
- Why does an external C memory checker need PYTHONMALLOC=malloc to be useful?Such a tool instruments the C library's allocation calls. With pymalloc in place it sees one 1 MiB arena where thousands of Python objects live, so per-object allocation sites are invisible and an overrun from one object into its neighbour never crosses a boundary the tool watches. Routing everything through malloc gives it one tracked allocation per object, at a significant slowdown.
- Would you ever set PYTHONMALLOC=malloc in production to reduce memory use?No. Small-object allocation gets slower without pymalloc's free lists, and retention does not improve — it simply becomes the system allocator's fragmentation instead of arena occupancy. It is a debugging configuration for chasing memory corruption in C extensions or an embedding, and a memory profile measured under it does not predict the default build's behaviour.
saying these in an interview costs you the question
- Calls PYTHONMALLOC a production performance tuning knob
- Thinks setting malloc reduces memory usage
- Expects an unknown value to fall back silently
- Confuses the debug hooks with a garbage collector setting
- Assumes allocator introspection is unaffected by the choice