Why can ast.literal_eval exhaust memory or CPU on untrusted input?

### The guarantee is narrow, and the narrowness is the point `ast.literal_eval` guarantees that **no code from the string runs**. It says nothing about how much time or memory the string costs before that guarantee is even relevant, because all of the work happens in the tokenizer and parser — which run to completion, building a full abstract syntax tree, *before* the whitelist walk ever looks at a node. An attacker who cannot execute code can still choose exactly how expensive your parse is. ### Measured on CPython 3.14 * **A flat literal is the real weapon.** A 6 MB source string of the form `[1,1,1,...]` with three million elements parses to a list in about **8.5 seconds** and peaks near **3 GB** of allocated memory — roughly 500 bytes of peak memory per source byte. A single request-sized payload can push a container past its memory limit. During most of that the work is in the C parser, so the thread holds the GIL and the rest of the process starves too. * **Deep nesting is already capped.** `"[" * 1000 + "]" * 1000` does not overflow anything; the tokenizer refuses at a couple of hundred levels with `SyntaxError: too many nested parentheses`. * **A long unary chain hits the parser stack.** `"-" * 100000 + "1"` raises `MemoryError: Parser stack overflowed - Python source too complex to parse` — a `MemoryError` from a string containing nothing but minus signs. * **Huge integer literals are capped too.** A 4400-digit integer raises `SyntaxError: Exceeds the limit (4300 digits) for integer string conversion`. That limit arrived in **3.11** as a denial-of-service fix (quadratic decimal-to-binary conversion) and is readable and settable at runtime with `sys.get_int_max_str_digits` and `sys.set_int_max_str_digits`. So CPython has closed the *stack* attacks and the *integer* attack, and left the plain allocate-a-lot attack wide open — as it must, since a large valid literal is indistinguishable from a large valid document. ### A concrete failure A translation-memory updater stores each entry's metadata as a Python literal string and re-reads it with `ast.literal_eval` during a nightly consolidation run that takes about six hours. A guard was added to skip oversized entries, but its boundary check was off by one at the batch edge, so one entry above the intended cap slipped through on every pass. Most nights nothing happened; the night an upstream import produced a multi-million-element list, the worker's RSS climbed past three gigabytes and the run died four hours in, having already written half its output. Nothing was executed, nothing was exploited, and the job still failed — which is exactly the point: the security property held and the availability property did not. ### Hardening the call site 1. **Cap the input before parsing.** `len(text.encode())` against an explicit ceiling is the only check that runs in constant time; every limit inside the parser fires after the cost is paid. Size the ceiling to your real data (kilobytes, usually), not to what feels generous. 2. **Catch the whole exception family** — `ValueError`, `SyntaxError`, `TypeError`, `MemoryError` and `RecursionError` — and re-raise one domain error. `except ValueError` alone lets `SyntaxError` and `MemoryError` escape into whatever handler is above you. 3. **Bound the blast radius.** Do the parsing in a worker process with an OS-level address-space limit and a timeout, so an unlucky payload kills one worker instead of the service. 4. **Do not let a `MemoryError` be caught and continued.** Once allocation has failed, the process is in an unreliable state; the honest response is to fail the unit of work and, usually, the process. 5. **Use a real parser for foreign data.** `json.loads` costs roughly 30x less for the same document and reports errors with a position. If the string is JSON, none of this is your problem to solve. ### What an interviewer is listening for Two sentences, really: *"`ast.literal_eval` protects against code execution, not against resource exhaustion,"* and *"so untrusted input gets a length cap before the call and a bounded worker around it."* Candidates who answer "it's safe, that's what it's for" have learned the marketing line; candidates who name the tokenizer nesting cap, the 4300-digit integer limit and the un-capped flat allocation have actually read what the function does.