skip to content

Why does CPython still push a frame when a Python function returns a call in tail position?

level: middleimportance: must knowfreq 50%

answer

  1. the compiler treats it like any call
  2. something has to receive the value
  3. frames are objects other tools can see
  4. tracebacks and tracing hooks read the chain
  5. 3.11 made calls cheap, not free

basics

~20 s

CPython performs no tail-call optimization: returning a call compiles to an ordinary call, so a new frame is pushed while the caller's frame stays alive to receive the result. Depth grows with every recursive step.

solid answer

~50 s

CPython has no tail-call optimization at any level. The compiler emits the same instructions for `return f(x)` as for `y = f(x); return y` — a call, then a separate return — so the interpreter pushes a fresh frame for the callee while the caller's frame waits underneath to receive the value. The refusal is deliberate. Frames in Python are reachable objects: a traceback carries one per level, `sys._getframe()` and `inspect.stack()` walk the live chain, and debuggers and profilers hook every call through `sys.settrace()`. Eliding the caller's frame would silently delete call history all of those report. Since 3.11 a Python-to-Python call no longer recurses into the C evaluator and frame objects are materialized lazily, so calls became much cheaper — but each call still pushes frame data, so depth still grows linearly. The remedy is to write the loop yourself.

code

python · 8 lines
python
import inspect

def countdown(n):
    if n == 0:
        return len(inspect.stack())
    return countdown(n - 1)          # tail position: nothing is reused

print(countdown(5) - countdown(0))   # 5 — one extra live frame per call

go deeper

for a junior

Be ready to state it plainly: Python does not optimize tail calls. A recursive call inside a return statement costs a frame like any other call, so the depth of the nesting grows with the size of the input.

for a middle

Explain the mechanics. The compiler emits an ordinary call followed by a separate return, the caller's frame stays alive to receive the value, and frames are real objects that tracebacks and the inspect module walk.

for a senior

Show what the refusal buys — truthful tracebacks, working debuggers, profilers and coverage tools — and be clear that the practical answer for deep input is a loop you write, not a flag you set.

for a principal

Own the tradeoff: a runtime can offer tail-call elimination or complete, honest stack introspection by default, not both. Argue why Python chose introspection and what that costs teams whose domain is naturally recursive.

### What "tail position" means A call sits in *tail position* when its value becomes the caller's return value and nothing is left to do afterwards. `return helper(n - 1)` is a tail call; `return 1 + helper(n - 1)` is not, because the addition still has to run once `helper` comes back. In a language that mandates tail-call elimination, the compiler notices there is no work left, reuses the caller's stack slot instead of adding one, and an arbitrarily long chain of tail calls runs in constant stack space. ### What CPython actually does CPython does not do this — at any optimization level, in any release, 3.14 included. The compiler emits exactly the same instruction sequence for a tail call as for any other call. Disassemble `def f(n): return f(n - 1)` and you see the function loaded, the argument computed, a call instruction, and then a *separate* return instruction. That separate return is the whole reason the caller must stay: the interpreter pushes a frame for the callee while the caller's frame sits underneath it holding the instruction pointer that says "resume at the return". Every recursive step adds another one, so the memory a recursion needs grows linearly with its depth, and the interpreter's nesting counter climbs on every call. ### Why the refusal is deliberate, not an oversight Frames in Python are not an invisible implementation detail; they are reachable objects that the language's own tooling depends on. A traceback carries one entry per active frame, which is what tells you the path the program took to the failure. `sys._getframe()` and `inspect.stack()` walk the live chain. Debuggers, profilers and coverage tools install a hook with `sys.settrace()` and are called on every call, line and return. Silently dropping the caller's frame on a tail call would delete call history that all of those report — a program would fail somewhere and the traceback would simply not mention the function that got you there. Python's designers have repeatedly judged that a faithful stack trace, every time, is worth more than unbounded recursion in a language where rewriting the recursion as a loop is easy. There is a mechanical argument too: the name in `return f(x)` is looked up dynamically at call time and can be rebound between calls, so a self-call is not statically guaranteed to be a self-call at all. ### What did change, and what people mistake for tail calls Two 3.x changes get misreported as tail-call optimization. In **3.11**, a Python function calling another Python function stopped recursing into the C evaluator. The interpreter pushes a lightweight frame record onto its own data stack and keeps running in the same C loop, and it materializes a full frame object lazily, only when something actually looks at it. Calls got substantially cheaper and stopped consuming C stack per Python call. Depth still grows: this is a cheaper frame, not a reused one. In **3.14**, CPython ships an optional build in which each bytecode handler is a C function that tail-calls the next handler instead of dispatching through a computed-goto table; a modern compiler turns those into jumps and the interpreter gets a few percent faster. That is the eval loop's own C-level calling convention. `return f(x)` in your Python code behaves exactly as it always did. The JIT is a separate experiment: off unless you set `PYTHON_JIT=1`, checkable with `sys._jit.is_available()`, with results ranging from somewhat slower to somewhat faster, and it adds no tail-call elimination either. ### What you do instead For direct self-recursion in tail position, write the loop by hand. Wrap the body in `while True:`, and where the recursive call was, rebind the parameter names to the new arguments and `continue`; where the base case was, `return`. That is the transformation elimination would have performed, done explicitly and visibly. For a recursion that branches — a walk over a nested structure — you need somewhere to keep the pending work, so carry an explicit stack: a list you seed with the root, pop from, and extend with children. Trampoline decorators exist as a curiosity: the wrapper loops, and the "recursive" call returns a small object describing the next call rather than making it. They handle only direct self-recursion, cost more per step than a plain loop, and destroy exactly the traceback and introspection that made CPython decline elimination in the first place. ### The interview answer Say the fact, then the mechanism, then the reason. Python has no tail-call optimization; `return f(x)` compiles to an ordinary call and the caller's frame stays alive to receive the value; frames are first-class objects that tracebacks, `inspect` and tracing hooks read, so eliding them would silently falsify the stack; and the practical remedy for deep input is a loop you write yourself.

  • Since 3.11 a Python-to-Python call no longer recurses into the C evaluator — doesn't that make deep tail recursion safe?
    No. That change removed C-stack recursion per Python call and made frame objects lazy, so calls got much cheaper. It did not make a frame reusable: the interpreter still pushes a frame record for every call and still counts the nesting, so memory grows linearly with depth and the nesting guard still fires. Faster recursion, not unbounded recursion.
  • CPython 3.14 has a tail-calling interpreter — is that tail-call optimization for Python code?
    No, and the name causes real confusion. It is a build-time change to how the evaluation loop dispatches: each bytecode handler is a C function that tail-calls the next one instead of jumping through a computed-goto table, which a modern compiler turns into plain jumps and which buys a few percent. Python-level `return f(x)` is completely unaffected.
  • Could a decorator give you tail-call elimination?
    Trampoline decorators exist: the wrapper runs a loop, and the recursive call returns a small object describing the next call rather than performing it. They handle only direct self-recursion, add per-call overhead, and destroy exactly the traceback and introspection that made CPython decline elimination in the first place. A hand-written `while` loop is simpler, faster and honest.

A tail call is like handing a job to a colleague and then standing in the doorway purely to pass their answer back out. CPython keeps you standing there even though you have nothing left to add.

saying these in an interview costs you the question

  • Claims CPython compiles a tail call into a jump
  • Thinks the -O flag or the JIT enables tail-call elimination
  • Confuses 3.14's tail-calling interpreter with Python-level TCO
  • Says the caller's frame is freed as soon as the tail call starts
  • Believes an accumulator parameter alone makes deep recursion safe

context