skip to content

Async Language Constructs

13 roadmaps12 questions

The syntax asyncio rides on: async for, async generators, async with, and the asyncio Lock, Event and Queue that replace their threading cousins. Interviewers use it to check for idiomatic async code.

part ofPythonoverview, primer and where to startread it →
on this pageshow

explore

questions

12

What does Python's `async with` statement do that a plain `with` cannot, and which methods does it call?

level: juniorimportance: must knowfreq 68%

answer

  1. Two protocols, not one
  2. Teardown that needs to do I/O
  3. Only legal inside async def
  4. A-prefixed pair of dunder methods
  5. Awaited result is bound, not the manager

basics

~10 s

async with awaits aenter before the block and awaits aexit after it, so setup and teardown can perform I/O. A plain with calls enter and exit synchronously and cannot await anything.

solid answer

~40 s

`with` runs the synchronous protocol: it calls `__enter__`, binds the result to the `as` target, and calls `__exit__(exc_type, exc, tb)` on the way out. Neither method can `await`, because they are ordinary function calls. `async with` runs the asynchronous protocol instead: it awaits `__aenter__()` and awaits `__aexit__(exc_type, exc, tb)`, so both halves may suspend and do I/O. That is the whole point — returning a connection to a pool, sending a close frame, committing a transaction or flushing a batch are themselves awaitable operations. `async with` is only legal inside an `async def`, `__aexit__` is awaited on every exit path including `return`, `break` and a propagating exception, and awaiting a truthy result from `__aexit__` suppresses that exception exactly as the synchronous protocol does. If `__aenter__` raises, `__aexit__` is never called.

code

python · 8 lines
python
from collections import deque

q = deque([1, 2, 3])
q.append(4)        # O(1) on the right
q.appendleft(0)    # O(1) on the left
print(q.popleft()) # 0
print(q.pop())     # 4
print(list(q))     # [1, 2, 3]

go deeper

for a junior

Be ready to name __aenter__ and __aexit__ on sight and to say why they exist: teardown that has to await. Know that async with only compiles inside an async def.

for a middle

Explain the mechanics: the awaited result of __aenter__ is what the as target binds, __aexit__ receives the exception triple, and a truthy awaited result suppresses the exception.

for a senior

Show the operational edges — __aexit__ running on every exit path, cancellation landing inside cleanup at an await point, and __aenter__ owning the rollback of anything it acquired before it failed.

for a principal

Own the API-design angle: whether a resource should expose an async context manager, a bare aclose(), or both, and what that choice costs callers who need to hold the resource across function boundaries.

## The two protocols Python has two context-manager protocols, and they are separate. The **synchronous** one is a pair of ordinary methods. `with expr as name:` evaluates `expr` to get a manager object, calls `manager.__enter__()`, binds its return value to `name`, runs the body, and then calls `manager.__exit__(exc_type, exc, tb)` — with three `None`s on a clean exit, or with the live exception's class, instance and traceback if the body raised. The **asynchronous** one is a pair of methods that return awaitables, `__aenter__` and `__aexit__`, and it is driven by `async with`, added in Python 3.5 by PEP 492 alongside `async def` and `await`. The statement does the same choreography, except that it *awaits* each call: it awaits `manager.__aenter__()`, binds the awaited result to the `as` target, runs the body, and awaits `manager.__aexit__(exc_type, exc, tb)`. In practice both methods are written as `async def`, which makes them return coroutine objects — but the language only requires that they return something awaitable. ## Why the async half has to exist A synchronous `__exit__` cannot `await`. It is invoked by the interpreter as a plain call in the middle of executing your coroutine's frame; there is no suspension point available inside it. So any teardown that is itself I/O has nowhere to go. And async teardown is the common case, not an exotic one. Returning a pooled connection often means writing a reset command; closing a client session means draining in-flight work and sending a shutdown frame; committing a transaction is a round trip; flushing a batched writer sends the last batch. All of those are awaitable. `async with` exists so that the cleanup you must do can be expressed where cleanup belongs, instead of being deferred to a `close()` the caller is expected to remember. The same argument applies to setup. `__aenter__` can await a handshake, wait for a slot in a semaphore, or acquire a lock that suspends rather than blocks — `async with lock:` on an `asyncio.Lock` is exactly that. ## The exact semantics worth stating in an interview * `async with` is a syntax error outside an `async def` body (or another asynchronous context such as an async generator or async comprehension scope). * The `as` target receives **the awaited result** of `__aenter__()`, not the manager. Most managers `return self`, which is why the distinction is easy to miss — but a manager is free to hand back something else entirely. * The `as` clause is optional; `async with lock:` is complete. * One statement may hold several managers, `async with a() as x, b() as y:`, entered left to right and exited right to left. Since 3.10 that list may be wrapped in parentheses and split across lines. * `__aexit__` is awaited on **every** exit path: falling off the end, `return`, `break`, `continue`, an exception propagating, and cancellation. * If awaiting `__aexit__` produces a truthy value, the exception that was passed in is swallowed and execution continues after the block. Returning `False` (or `None`, by falling off the end) lets it propagate. Suppressing by accident — for example by ending `__aexit__` with an expression that happens to be truthy — is a real bug class. * If `__aenter__` raises, the block never runs and `__aexit__` is **not** called, exactly as in the synchronous protocol. Anything `__aenter__` acquired before failing must be cleaned up by `__aenter__` itself. * Because `__aexit__` is awaited, it contains suspension points, and a task cancelled during teardown can raise `asyncio.CancelledError` out of the middle of your cleanup. Cleanup that must complete has to be protected, for example with `asyncio.shield`. ## Standard-library examples The asynchronous protocol is not niche: `asyncio.Lock`, `asyncio.Semaphore` and the other asyncio primitives are async context managers; `asyncio.TaskGroup` and `asyncio.timeout` (both 3.11) are async context managers whose entire behaviour lives in `__aexit__`; `contextlib.aclosing` (3.10) wraps an object with an `aclose()` coroutine method; `contextlib.nullcontext` has supported the async protocol since 3.10. ## Mismatch is loud in one direction only Using `with` on an object that implements only the async protocol raises `TypeError` at the `with` statement, and CPython's message says so and suggests `async with`. The reverse mistake — a class whose `__exit__` is accidentally an `async def`, used under a plain `with` — is silent and much worse, which is a separate discussion in its own right. The one-line summary an interviewer wants: **`with` is `__enter__`/`__exit__` and cannot await; `async with` is `__aenter__`/`__aexit__` and awaits both, so setup and teardown may do I/O.**

  • What is bound to the name after `as` in `async with open_feed() as feed:`?
    The awaited result of `__aenter__()`. The statement awaits the coroutine `__aenter__` returns and binds that value. Most managers `return self`, so it usually looks like the manager itself, but nothing requires that — a manager can enter a resource and hand back a cursor, a session or a plain tuple.
  • If `__aexit__` awaits successfully but returns True, what happens to an exception raised in the block?
    It is suppressed. The truthy result tells the statement the exception has been handled, so execution resumes after the `async with` block and the exception never propagates. This is the same rule as synchronous `__exit__`, and it is why cleanup methods should end with an explicit `return False` or simply fall off the end rather than returning whatever the last expression evaluated to.
  • Is `__aexit__` still awaited when the block exits via `return` or an unhandled exception?
    Yes, on every exit path — falling off the end, `return`, `break`, `continue`, a propagating exception, and cancellation. The only case where it is not called is when `__aenter__` itself raised, because the manager was never successfully entered; anything `__aenter__` acquired before failing must be released inside `__aenter__`.

A plain with is a door that latches shut the instant you let go. async with is a door with a closer that needs a moment to run — the statement waits for it before moving on.

saying these in an interview costs you the question

  • Says `async with` is just `with` inside async code
  • Claims `__enter__` can await if the function is async
  • Thinks the `as` name is always the manager object
  • Believes `__aexit__` is skipped when the block raises
  • Cannot name `__aenter__` and `__aexit__` at all

context

resolving links…

open as a page

Why use asyncio.Lock instead of threading.Lock inside a coroutine?

level: juniorimportance: must knowfreq 58%

basics

~20 s

asyncio.Lock suspends only the awaiting task, so the event loop keeps running everything else. threading.Lock blocks the entire thread the loop runs on, freezing every task on that loop — including the one that would release the lock.

open as a page

What must an object implement for Python's `async for` to iterate it?

level: juniorimportance: must knowfreq 60%

basics

~10 s

Its type must define __aiter__, an ordinary method returning an async iterator, and that iterator's type must define __anext__, a coroutine that resolves to the next item or raises StopAsyncIteration when the stream ends.

open as a page

Why can't `__init__` await, and where must an async resource wrapper open its connection?

level: middleimportance: must knowfreq 55%

basics

~20 s

Construction is synchronous: __init__ must return None, so an async def __init__ returns a coroutine and raises TypeError. Put the awaited connect in __aenter__ and the awaited close in __aexit__, and use the object under async with.

open as a page

How does asyncio.Semaphore bound a fan-out of hundreds of coroutines?

level: middleimportance: must knowfreq 55%

basics

~20 s

asyncio.Semaphore(n) holds n permits. async with sem: takes one and suspends the task when none are left, so at most n coroutines are inside the guarded block at once; waiters are released in order as permits come back.

open as a page

How does an `async def` containing `yield` differ from a coroutine function?

level: middleimportance: must knowfreq 55%

basics

~20 s

It is an async generator function: calling it returns an async generator object, not a coroutine, and runs no code. You never await that object — you drive it with async for or anext(), and it may not return a value.

open as a page

When does `contextlib.AsyncExitStack` beat nested `async with` blocks, and how do you register cleanup on it?

level: middleimportance: should knowfreq 32%

basics

~20 s

Use AsyncExitStack when the number of resources is decided at runtime or some are conditional, since nested async with needs a fixed, statically written shape. Register with enter_async_context, push_async_callback or enter_context; everything unwinds in reverse order.

open as a page

A `with` block whose `__exit__` is an `async def` silently swallows errors. Why, and how do you catch it?

level: seniorimportance: should knowfreq 38%

basics

~20 s

A plain with calls exit and gets back a coroutine object it never awaits, so cleanup never runs. Coroutine objects are truthy, and a truthy exit result means suppress, so exceptions vanish. Only a RuntimeWarning marks it.

open as a page

asyncio.Queue with no maxsize grows without bound in a streaming job — how do you apply backpressure?

level: seniorimportance: should knowfreq 40%

basics

~20 s

An asyncio.Queue defaults to maxsize=0, meaning unbounded, so a fast producer buffers the whole stream in memory. Give it a maxsize: once full, await queue.put(...) suspends the producer until a consumer drains an item, which is backpressure.

open as a page

Why might an async generator's `finally` cleanup not run when a consumer breaks out of the `async for`, and how do you make it deterministic?

level: seniorimportance: should knowfreq 45%

basics

~20 s

break leaves the generator suspended at its yield, so its finally runs only when something closes it — at garbage collection, scheduled onto the event loop, or at the loop's shutdown sweep. Wrap it in contextlib.aclosing to close it on every exit path.

open as a page

What does asyncio.Queue.join() wait for, and what must consumers call?

level: middleimportance: nice to knowfreq 32%

basics

~10 s

join() waits until the queue's unfinished-items counter reaches zero. Every put increments it, and only a consumer calling task_done() decrements it. Consumers that get items but never call task_done() leave join() waiting forever.

open as a page

How do you collect an async generator's values into a list in Python?

level: middleimportance: nice to knowfreq 30%

basics

~20 s

With an async comprehension, [item async for item in stream()], written inside an async def. list() and sorted() raise TypeError on an async generator because they need __iter__, and the standard library has no helper that materializes one.

open as a page