skip to content

When does `contextlib.AsyncExitStack` beat nested `async with` blocks, and how do you register cleanup on it?

level: middleimportance: should knowfreq 32%

answer

  1. Nesting is fixed at authoring time
  2. Some resources are optional or counted
  3. One holder, many registered exits
  4. Reverse order on the way out
  5. Ownership can be transferred out

basics

~20 s

Use AsyncExitStack when the number of resources is decided at runtime or some are conditional, since nested async with needs a fixed, statically written shape. Register with enter_async_context, push_async_callback or enter_context; everything unwinds in reverse order.

solid answer

~40 s

Nested `async with` is fine when you know at authoring time exactly which managers you need. `contextlib.AsyncExitStack` is for when you do not: a count decided by configuration, optional resources behind feature flags, or a factory that acquires several things and hands the caller a live bundle. It is itself an async context manager. `await stack.enter_async_context(cm)` awaits `cm.__aenter__()`, registers its `__aexit__`, and returns the `as` value; `stack.enter_context(cm)` does the same for a synchronous manager; `stack.push_async_callback(fn, *args)` registers a coroutine function to be awaited on the way out. Unwinding is strictly LIFO and behaves exactly like the equivalent nesting, including exception propagation. `stack.pop_all()` transfers everything to a fresh stack so a half-built bundle can be returned without being torn down.

code

python · 9 lines
python
from collections import deque

d = deque([1, 2, 3])
d.extendleft([4, 5, 6])
print(list(d))   # [6, 5, 4, 1, 2, 3]

d = deque([1, 2, 3])
d.extendleft(reversed([4, 5, 6]))
print(list(d))   # [4, 5, 6, 1, 2, 3]

go deeper

for a junior

Know that it exists and what it is for: a runtime-decided number of resources cleaned up together, instead of a fixed pyramid of nested blocks you would have to write by hand.

for a middle

Be able to write it — await stack.enter_async_context(...) in a loop, push_async_callback for extra teardown, enter_context for a synchronous manager — and state that unwinding is reverse order.

for a senior

Show judgement about when nesting is clearer, explain that acquisition is sequential and how you would overlap it, and be able to reason about exceptions raised during the unwind.

for a principal

Own the resource-ownership contract at API boundaries: which layer holds the stack, whether factories hand back live bundles via pop_all(), and how partial-acquisition failure is guaranteed not to leak.

## The problem it solves `async with a() as x:` nested inside `async with b() as y:` is a **static** construction: the number of managers, and which ones, is fixed when you write the code. Real programs often decide at runtime. A museum-catalogue importer might open one feed session per source institution, where the list of institutions comes from configuration; an 11-person team's staging config might list three, production eleven. You cannot write eleven nested blocks, and you certainly cannot write "however many the config says". `contextlib.AsyncExitStack`, added in Python 3.7, is the asynchronous counterpart of `contextlib.ExitStack`. It is a single async context manager that holds a stack of other managers and callbacks, entered as you go and unwound in reverse when the stack exits. ```python async with AsyncExitStack() as stack: feeds = [await stack.enter_async_context(source(n)) for n in names] ... # every feed closed here, last opened first ``` ## The registration methods * **`await stack.enter_async_context(cm)`** — awaits `cm.__aenter__()`, records `cm.__aexit__` for the unwind, and returns the value `__aenter__` produced. This is the workhorse; note it is a coroutine, so it must be awaited. * **`stack.enter_context(cm)`** — the same for a *synchronous* context manager. An `AsyncExitStack` can hold both kinds, which matters when a task mixes a file handle with a network session. * **`stack.push_async_callback(fn, *args, **kwargs)`** — registers a coroutine function to be awaited during unwind. It receives no exception information and its result is ignored, so it can never suppress; use it for "flush the index", "emit a metric", "release the reservation". * **`stack.callback(fn, *args, **kwargs)`** — the synchronous equivalent of the above. * **`stack.push_async_exit(exit)`** — registers something with the `__aexit__` signature (an object that has one, or an async callable shaped like it). Unlike `push_async_callback`, it *does* see the exception and *can* suppress it. * **`stack.push(exit)`** — the synchronous form of `push_async_exit`. * **`await stack.aclose()`** — unwind immediately and explicitly, instead of relying on the surrounding `async with`. ## Ordering and exceptions Unwinding is strictly last-in-first-out, and the semantics are deliberately identical to the equivalent nesting: each registered exit sees the exception currently in flight, may suppress it by returning a truthy value, and may raise a new one — in which case the original is attached as its context, exactly as a raise inside a nested `__aexit__` would be. If several cleanups raise, the last one propagates with the earlier ones chained. Reasoning about a stack is therefore no harder than reasoning about the nesting you would have written by hand. One thing the stack does **not** do is parallelise acquisition. Each `enter_async_context` is awaited in turn, so eleven feeds that take 200 ms each to open cost 2.2 seconds. If setup latency matters, open them concurrently — for example inside an `asyncio.TaskGroup` (3.11) — and then register the teardowns on the stack with `push_async_callback`. ## `pop_all()` — the all-or-nothing acquisition pattern The subtlest and most useful method. A factory that acquires several resources wants either to hand back all of them, live, or to release everything it managed to acquire if a later step fails. That is exactly: ```python async def open_bundle(names): async with AsyncExitStack() as stack: feeds = [await stack.enter_async_context(source(n)) for n in names] await verify(feeds) # may raise: stack unwinds, all released return feeds, stack.pop_all() # success: ownership transferred out ``` `pop_all()` moves every registered exit to a **new** `AsyncExitStack` and empties the old one, so when the `async with` finishes it has nothing left to close. The caller now owns the returned stack and is responsible for `aclose()`-ing it or entering it. Without `pop_all()` there is no clean way to write a partial-failure-safe multi-resource factory. ## When *not* to reach for it If the managers are known and few, nested `async with` (or one header with several comma-separated managers, parenthesized across lines since 3.10) is clearer and needs no explanation. `AsyncExitStack` earns its keep when the shape is dynamic, conditional, or must escape the function that built it — and it costs a reader a moment of unfamiliarity everywhere else. ## The interview summary Dynamic or conditional resource sets; `enter_async_context` for async managers and `enter_context` for sync ones on the same stack; `push_async_callback` for fire-and-forget teardown that cannot suppress; LIFO unwinding with nesting-identical exception semantics; `pop_all()` to hand a fully-acquired bundle to a caller.

  • What does `stack.pop_all()` do and why would a factory function need it?
    It moves every registered exit onto a fresh `AsyncExitStack` and empties the original, so the surrounding `async with` closes nothing. A factory can therefore acquire several resources under a stack — releasing all of them automatically if any step fails — and on success return the bundle plus the new stack, handing ownership to the caller instead of tearing everything down on return.
  • How do `push_async_callback` and `push_async_exit` differ?
    `push_async_callback` registers a plain coroutine function that is awaited with the arguments you gave it; it receives no exception information and its result is ignored, so it cannot suppress anything. `push_async_exit` registers a callable or object with the `__aexit__` signature, so it does receive the exception triple and can suppress by returning a truthy value. Use the callback form for fire-and-forget teardown.
  • Does `AsyncExitStack` open its resources concurrently?
    No. Each `enter_async_context` is awaited in sequence, so acquisition latency adds up. When setup cost matters, open the resources concurrently — for example under an `asyncio.TaskGroup` — and then register their teardowns on the stack with `push_async_callback` or `push_async_exit` so the unwinding is still centralised and ordered.

saying these in an interview costs you the question

  • Reaches for it when the manager set is fixed and small
  • Forgets to await enter_async_context
  • Expects registration order rather than reverse on unwind
  • Thinks it can only hold asynchronous context managers
  • Assumes it acquires the resources concurrently
  • Cannot explain how a factory returns live resources safely

context