Why can raising sys.setrecursionlimit crash the process instead of curing RecursionError?

## The counter and the stack are different things `sys.setrecursionlimit(n)` writes one integer into the interpreter. Nothing is allocated, nothing is reserved, no thread is reconfigured. The only effect is that CPython's per-thread frame-depth counter is now compared against a larger number before it refuses another Python call. The resource that can actually run out is the **C stack**: the fixed-size memory region the operating system gives a thread when it is created, commonly around 8 MiB for a process's main thread and often smaller for threads created afterwards. Overrunning it is not an exception — the OS faults the process. The recursion limit was introduced as a cheap proxy that trips *before* that happens. Raising it moves the guardrail toward the edge; raise it far enough and you have removed the guardrail entirely while leaving the edge where it was. ## What changed, and why the answer is not simply "never raise it" The relationship between Python frames and C stack frames has changed materially: * **3.11** inlined Python-to-Python calls in the evaluation loop. A pure-Python function calling another pure-Python function no longer costs a nested C-level call, so depth in that shape is now bounded mostly by heap memory for frames rather than by the thread stack. * **3.12** split the accounting outright. `sys.setrecursionlimit` governs the Python-frame budget; recursion that goes *through* C code is counted separately, against an internal limit that Python code cannot raise. * **3.13 and 3.14** continued to harden the C-side check so that more near-overflow situations surface as `RecursionError` rather than as a crash. Treat that as best-effort hardening, not a guarantee. So on 3.14, `sys.setrecursionlimit(50_000)` around a plain recursive walk over pure-Python functions is far more likely to work than the same call on 3.10. The trap is that most real recursion is not purely Python-to-Python. ## The C-mediated cases that still bite Each of these re-enters the interpreter from C once per level, so each level costs real stack: * a `__repr__` or `__str__` that formats a nested structure, since formatting runs in C between the levels; * equality or ordering comparisons on deeply nested containers, where the container's comparison is C code that calls back into yours; * deep-copying or serializing a deeply nested structure through a C-implemented codec; * a recursive descent driven by a C-level callback, where each level goes user code → C → user code. For these, the number you raised is not the binding constraint, and pushing it higher does not buy depth — it only removes the polite failure. ## The crash is worse than the exception A `RecursionError` is an ordinary exception: the stack unwinds, `finally` blocks run, context managers exit, buffers flush, your logger records what happened and which input caused it. A stack overflow is a signal: the process vanishes mid-operation. Anything holding a lock, a half-written file, or an in-flight batch is left exactly as it was. In a long-running service, one is a bad request and the other is an incident. A raised limit also costs memory even when it works, because every live frame retains its locals. Deep recursion over structures that hold large values keeps all of them reachable until the stack unwinds, which is how an ordinary-looking walk turns into a working set several times the size of the data being walked. ## Practical rules 1. **Diagnose before you raise.** If the recursion is unbounded — a cycle or a missing base case — no limit value fixes it; you have only made the failure slower and more expensive. 2. **Save and restore.** The setting is interpreter-wide with no scoping and no context manager, so a library that leaves it raised has changed the failure mode of every other component in the process. ```python import sys previous = sys.getrecursionlimit() sys.setrecursionlimit(20_000) try: ... finally: sys.setrecursionlimit(previous) ``` 3. **Do not lower it carelessly either.** Setting a limit below the depth already in use raises `RecursionError` immediately, which is a surprising way to break a restore path that runs inside a deep call chain. 4. **Prefer an explicit bound.** A traversal that counts its own depth and rejects input past a documented maximum fails predictably at a place you chose, with a message naming the offending record — much better than an interpreter guard firing wherever it happens to land. 5. **If you genuinely need real depth**, the honest levers are a bigger stack for the thread that runs the work, or restructuring so depth lives in a heap data structure instead of the call stack. Raising the counter alone is the one lever that changes the failure mode without changing the capacity.