skip to content

Async Iteration and Async Generators

async for drives the __aiter__/__anext__ protocol, and an async def containing yield is an async generator — the natural way to stream paginated results. Interviewers probe finalization and aclose.

part ofPythonoverview, primer and where to startread it →
on this pageshow

questions

4

What must an object implement for Python's `async for` to iterate it?

level: juniorimportance: must knowfreq 60%

answer

  1. Two methods, not one
  2. A loop that can wait between items
  3. The stream ends with its own exception
  4. __aiter__ synchronous, __anext__ awaited
  5. StopAsyncIteration, never StopIteration

basics

~10 s

Its type must define __aiter__, an ordinary method returning an async iterator, and that iterator's type must define __anext__, a coroutine that resolves to the next item or raises StopAsyncIteration when the stream ends.

solid answer

~50 s

`async for` runs on a two-method protocol that mirrors `for`. The object being iterated exposes `__aiter__` on its type — a plain, non-`async` method that hands back an async iterator. That iterator exposes `__anext__`, a coroutine function; each turn of the loop awaits it, and it either resolves to the next item or raises `StopAsyncIteration` to end the loop. The separate exception exists because `StopIteration` cannot travel out of a coroutine. Because the loop awaits, `async for` is only legal inside an `async def` body, and it is sequential: it lets other tasks run *while* an item is being fetched, but it does not fetch items in parallel. Most code never writes the two methods by hand — an async generator supplies them — but a class wrapping a paginated feed or a database cursor implements them directly.

code

python · 21 lines
python
import asyncio

class Countdown:
    def __init__(self, start):
        self.remaining = start

    def __aiter__(self):
        return self

    async def __anext__(self):
        if self.remaining == 0:
            raise StopAsyncIteration
        self.remaining -= 1
        await asyncio.sleep(0)
        return self.remaining

async def main():
    async for value in Countdown(3):
        print(value)

asyncio.run(main())

go deeper

for a junior

Be ready to name both methods — __aiter__ and __anext__ — and the exception that ends the loop, and to say that async for is only legal inside an async def body.

for a middle

Explain the desugaring out loud: __aiter__ is called once to get the iterator, then __anext__ is awaited each turn until StopAsyncIteration. Say why the synchronous protocol could not simply be reused.

for a senior

Show when you hand-write the protocol instead of reaching for an async generator — a cursor or paging client that owns connection state — and how you keep a sequential loop from becoming the latency floor of a request.

for a principal

Own the API decision: exposing a stream as an async iterator commits every consumer to async def and to sequential consumption. Be able to argue when a callback, a queue hand-off, or a batch-returning call serves consumers better.

### The protocol, stated exactly `async for` is the asynchronous counterpart of `for`, and it runs on a two-method protocol that mirrors the synchronous one method for method. * An **async iterable** is an object whose *type* defines `__aiter__`. On Python 3.14 that is an ordinary, non-`async` method, and it returns an **async iterator**. * An **async iterator** is an object whose type defines `__anext__`. `__anext__` is a coroutine function: calling it returns an awaitable, and awaiting that awaitable either produces the next item or raises `StopAsyncIteration`. Both names are looked up on the type, not on the instance, exactly like `__iter__` and `__next__` — assigning a function to an instance attribute does not make the object iterable. ### What the loop compiles to ```python iterator = type(source).__aiter__(source) while True: try: item = await type(iterator).__anext__(iterator) except StopAsyncIteration: break # loop body ``` Two consequences fall straight out. First, the loop contains an `await`, so `async for` is only legal inside an `async def` body; at module level or in a plain `def` it is a `SyntaxError`. Second, the loop awaits **one** item at a time. `async for` lets the event loop run other work *while* an item is being produced, but the items themselves arrive strictly in sequence — it is not a parallel map. Overlapping the fetches means creating tasks and gathering them, not writing a tighter loop. ### Why the protocol exists at all The synchronous protocol cannot suspend. `__next__` must return a value or raise, right now, on the calling thread; if producing the next item needs a network round trip, the thread blocks and the event loop stops. `__anext__` returns an awaitable instead, so the point where the next item is produced becomes a suspension point like any other. That single change is what lets an HTTP response body, a database cursor, a message consumer and a chunked file reader all present themselves as ordinary loops. ### Why a separate exception `StopIteration` is already load-bearing inside coroutines: it is how a generator-based coroutine signals its return value, and CPython converts a `StopIteration` that escapes a coroutine into a `RuntimeError`. Reusing it for "the stream ended" would have made end-of-iteration indistinguishable from a coroutine returning. `StopAsyncIteration` is a builtin exception added with the protocol for exactly that reason. The symmetry holds in the other direction too: raising `StopIteration` inside `__anext__` does not end your `async for` — it surfaces as a `RuntimeError`. ### The version detail that still catches people Early releases allowed `__aiter__` to be a coroutine function whose awaited result was the iterator. That transitional form was deprecated and **removed in 3.8**. On 3.14, writing `async def __aiter__` earns a `TypeError` saying the object returned from `__aiter__` does not implement `__anext__` — because what came back was a coroutine object. `__aiter__` stays synchronous; `__anext__` is the half that awaits. ### The builtins and the ABCs Python 3.10 added `aiter()` and `anext()`, the async twins of `iter()` and `next()`. `aiter(source)` calls `__aiter__`; `await anext(iterator)` calls and awaits `__anext__`; and `await anext(iterator, default)` returns `default` instead of raising `StopAsyncIteration`, which is how you take just the first item of a stream without writing a loop. `collections.abc` carries `AsyncIterable`, `AsyncIterator` and `AsyncGenerator` for isinstance checks and annotations; subclassing `AsyncIterator` supplies an `__aiter__` that returns `self`, so a hand-written iterator only has to provide `__anext__`. ### When you write the two methods by hand Most streaming code never does — an `async def` containing `yield` generates the protocol for you. You reach for the class form when the iterator must own state that outlives one pass, expose more than iteration (a page token, a cursor position, an explicit close), or be re-iterable: `__aiter__` returning a **fresh** iterator each call makes the object re-iterable, while returning `self` makes it one-shot and shared. A paging client is the textbook case — `__anext__` serves buffered records until the buffer empties, then awaits the next page request and refills it. ### Failure modes to recognize * A plain `for` over an async iterator raises `TypeError`; the two protocols do not fall back to one another in either direction. * `async for` over a list raises `TypeError`, because `list` has no `__aiter__`. * Two consumers sharing one async iterator do not each see every item — every `__anext__` advances the same state, so the items are split between them. * Leaving the loop early does not finish the iterator; if it holds a resource, that resource stays held until something closes it. Getting this protocol right is the entry ticket to every streaming API in async Python, and the four lines of desugaring above answer most of the questions that follow from it.

  • Why did async iteration need `StopAsyncIteration` instead of reusing `StopIteration`?
    `StopIteration` already means something inside a coroutine: it is how a generator-based coroutine delivers its return value, and CPython turns a `StopIteration` that escapes a coroutine into a `RuntimeError`. If it also meant end-of-stream, a coroutine returning normally would be indistinguishable from an exhausted iterator. A distinct builtin exception keeps the two protocols from colliding.
  • Does an `async for` loop fetch its items concurrently?
    No. It awaits `__anext__` once per turn, so items are produced strictly in order; the concurrency it buys is that other tasks run while each item is in flight. To overlap the work itself you need one task per unit of work and something that gathers the results — the loop shape alone will never do it.
  • What do the `aiter()` and `anext()` builtins give you over calling the dunders?
    They are the async twins of `iter()` and `next()`, added in 3.10. `aiter(source)` invokes `__aiter__` with proper type-level lookup, and `await anext(iterator)` invokes and awaits `__anext__`. The two-argument form, `await anext(iterator, default)`, returns the default instead of raising `StopAsyncIteration`, which is the clean way to peek at the first item without a loop.

A for loop reads from a bucket that is already full; an async for loop asks for each item and lets other work run while that item is being fetched.

saying these in an interview costs you the question

  • Claims `async for` fetches its items in parallel
  • Expects `StopIteration` to end an `async for` loop
  • Writes `__aiter__` as `async def` and awaits its result
  • Thinks any awaitable can be used with `async for`
  • Uses a plain `for` over an async iterator and expects it to work

context

open as a page

How does an `async def` containing `yield` differ from a coroutine function?

level: middleimportance: must knowfreq 55%

basics

~20 s

It is an async generator function: calling it returns an async generator object, not a coroutine, and runs no code. You never await that object — you drive it with async for or anext(), and it may not return a value.

open as a page

Why might an async generator's `finally` cleanup not run when a consumer breaks out of the `async for`, and how do you make it deterministic?

level: seniorimportance: should knowfreq 45%

basics

~20 s

break leaves the generator suspended at its yield, so its finally runs only when something closes it — at garbage collection, scheduled onto the event loop, or at the loop's shutdown sweep. Wrap it in contextlib.aclosing to close it on every exit path.

open as a page

How do you collect an async generator's values into a list in Python?

level: middleimportance: nice to knowfreq 30%

basics

~20 s

With an async comprehension, [item async for item in stream()], written inside an async def. list() and sorted() raise TypeError on an async generator because they need __iter__, and the standard library has no helper that materializes one.

open as a page