How do asyncio's StreamReader.read, readline and readexactly differ at end of stream?

middleshould knowfreq 42%

answer

Four methods, four contracts
One of them forgives a truncated tail
Two of them raise instead
The raised error carries what it got
A delimiter search needs a cap

basics

~20 s

read returns whatever is available, and empty bytes at EOF. readline returns a partial line without its newline at EOF, never raising. readexactly and readuntil raise asyncio.IncompleteReadError when the stream ends early, carrying the bytes already read.

solid answer

~50 s

await reader.read(n) returns up to n bytes and b"" once the peer has closed; read() with no argument reads until EOF. await reader.readline() reads through the next \n; if EOF arrives first it returns the partial line silently, without the newline, and returns b"" when nothing was buffered — so a truncated last line looks exactly like a valid unterminated one. await reader.readuntil(sep) and await reader.readexactly(n) are the strict pair: both raise asyncio.IncompleteReadError on an early EOF, and the exception carries what was read in its partial attribute and, for readexactly, the count it wanted in expected. That is why length-prefixed framing uses readexactly: short frames raise instead of being mistaken for whole ones. Also note the 64 KiB stream limit — readline raises ValueError and readuntil raises asyncio.LimitOverrunError when a single line exceeds it, while readexactly is exempt.

`await reader.read(n)` returns **up to** n bytes and `b""` once the peer has closed; `read()` with no argument reads until EOF. `await reader.readline()` reads through the next `\n`; if EOF arrives first it **returns the partial line silently**, without the newline, and returns `b""` when nothing was buffered — so a truncated last line looks exactly like a valid unterminated one. `await reader.readuntil(sep)` and `await reader.readexactly(n)` are the strict pair: both raise `asyncio.IncompleteReadError` on an early EOF, and the exception carries what was read in its `partial` attribute and, for `readexactly`, the count it wanted in `expected`. That is why length-prefixed framing uses `readexactly`: short frames raise instead of being mistaken for whole ones. Also note the 64 KiB stream limit — `readline` raises `ValueError` and `readuntil` raises `asyncio.LimitOverrunError` when a single line exceeds it, while `readexactly` is exempt.

code

python · 13 lines

python

import asyncio

async def main():
    reader = asyncio.StreamReader()
    reader.feed_data(b"\x00\x00\x00\x05hel")
    reader.feed_eof()
    length = int.from_bytes(await reader.readexactly(4), "big")
    try:
        await reader.readexactly(length)
    except asyncio.IncompleteReadError as exc:
        print(f"wanted {exc.expected}, got {len(exc.partial)}: {exc.partial!r}")

asyncio.run(main())

go deeper

for a junior

Know that a read can return fewer bytes than you asked for and that empty bytes mean the peer closed. Be able to loop until EOF without assuming message boundaries.

for a middle

Contrast the four methods' end-of-stream behaviour: read returns short then empty, readline returns a partial line silently, readuntil and readexactly raise IncompleteReadError carrying the partial bytes.

for a senior

Show you have debugged a truncation: pick length-prefixed framing with readexactly for binary protocols, justify the buffering limit as a denial-of-service control, and log the expected-versus-received counts from the exception.

for a principal

Own the framing decision across services — delimiter versus length prefix versus an existing framed protocol — and the compatibility and observability consequences of changing it once peers are deployed.

Framing is the actual subject

A stream is a byte pipe with no message boundaries. Whatever structure your messages have, you impose it — and the four read methods on asyncio.StreamReader are four different framing contracts with four different end-of-stream behaviours. Getting them mixed up produces the classic off-by-one truncation bug: the last record of a scrape looks complete, is one byte short, and is quietly accepted.

read(n) — no framing at all

await reader.read(n) returns at most n bytes: whatever is buffered when it is called, or the first chunk to arrive if the buffer is empty. It never waits to fill n. Once the peer has closed and the buffer is drained it returns b"" forever, which is how you detect EOF. read() with the default argument of -1 means "read until EOF", accumulating everything — fine for a small response, a memory hazard for a large one.

The mistake to avoid: treating one read() as one message. Two writes from the peer can arrive coalesced in one read, and one write can be split across several. TCP preserves order, not boundaries.

readline() — delimiter framing, lenient at EOF

await reader.readline() reads up to and including the next \n and returns it with the newline attached. Its EOF rule is the one people get wrong: if the stream ends before a newline arrives, it returns the partial data without raising, newline absent. If nothing was buffered at all it returns b"", which is the EOF signal.

So a truncated final line is indistinguishable from a legitimately unterminated one at the API level. If truncation matters — and in a metrics scraper it does, because half a sample line parses into a plausible wrong number — you must check the return value yourself: a line that does not end in \n and is not empty means the stream died mid-line.

Internally readline is readuntil(b"\n") with the strictness removed: it catches IncompleteReadError and returns the exception's partial bytes.

readuntil(sep) — delimiter framing, strict at EOF

await reader.readuntil(separator) is the same idea without the forgiveness: an early EOF raises asyncio.IncompleteReadError. Since Python 3.13 the separator may be a tuple of separators, in which case the shortest match wins — useful for a protocol that accepts both \n and \r\n.

readexactly(n) — length framing

await reader.readexactly(n) waits until exactly n bytes are available and returns them, or raises asyncio.IncompleteReadError if EOF comes first. The exception carries the bytes it did manage to read in its partial attribute and the count it was waiting for in expected, which makes the log line for a truncated frame trivially precise.

This is the method for length-prefixed protocols: read a fixed-size header with readexactly, decode the length from it, then readexactly that many body bytes. A short frame raises instead of being handed to your parser as if it were whole. readexactly is deliberately not subject to the stream's limit, because the length came from the protocol, not from a delimiter search.

The limit, and which exception it raises

A StreamReader is created with a limit, 64 KiB by default, which caps how much may be buffered while hunting for a delimiter — otherwise a peer that never sends \n would make you buffer forever, which is a denial-of-service primitive as much as a bug. When a single line exceeds it, readuntil raises asyncio.LimitOverrunError and leaves the data in the buffer, while readline catches that and re-raises it as a plain ValueError after discarding. Both asyncio.open_connection and asyncio.start_server take a limit keyword argument if your protocol legitimately has longer lines.

Choosing

You control the protocol and messages can be large or binary → length prefix plus readexactly. Cheapest to parse, strict about truncation, no escaping needed.
The protocol is line-oriented and text → readline, plus an explicit check that the line ended in \n if truncation must be detected; or readuntil if you would rather it raised.
You are proxying opaque bytes and boundaries do not matter → read(n) in a loop until b"".

What interviewers listen for

That read(n) may return fewer than n bytes; that readline swallows truncation while readexactly and readuntil raise; and that IncompleteReadError carries the partial data so the failure can be reported precisely rather than as a bare parse error three layers up.

## Framing is the actual subject A stream is a byte pipe with no message boundaries. Whatever structure your messages have, you impose it — and the four read methods on `asyncio.StreamReader` are four different framing contracts with four different end-of-stream behaviours. Getting them mixed up produces the classic off-by-one truncation bug: the last record of a scrape looks complete, is one byte short, and is quietly accepted. ## read(n) — no framing at all `await reader.read(n)` returns **at most** n bytes: whatever is buffered when it is called, or the first chunk to arrive if the buffer is empty. It never waits to fill n. Once the peer has closed and the buffer is drained it returns `b""` forever, which is how you detect EOF. `read()` with the default argument of -1 means "read until EOF", accumulating everything — fine for a small response, a memory hazard for a large one. The mistake to avoid: treating one `read()` as one message. Two writes from the peer can arrive coalesced in one read, and one write can be split across several. TCP preserves order, not boundaries. ## readline() — delimiter framing, lenient at EOF `await reader.readline()` reads up to and including the next `\n` and returns it with the newline attached. Its EOF rule is the one people get wrong: **if the stream ends before a newline arrives, it returns the partial data without raising**, newline absent. If nothing was buffered at all it returns `b""`, which is the EOF signal. So a truncated final line is indistinguishable from a legitimately unterminated one at the API level. If truncation matters — and in a metrics scraper it does, because half a sample line parses into a plausible wrong number — you must check the return value yourself: a line that does not end in `\n` and is not empty means the stream died mid-line. Internally `readline` is `readuntil(b"\n")` with the strictness removed: it catches `IncompleteReadError` and returns the exception's partial bytes. ## readuntil(sep) — delimiter framing, strict at EOF `await reader.readuntil(separator)` is the same idea without the forgiveness: an early EOF raises `asyncio.IncompleteReadError`. Since Python 3.13 the separator may be a **tuple** of separators, in which case the shortest match wins — useful for a protocol that accepts both `\n` and `\r\n`. ## readexactly(n) — length framing `await reader.readexactly(n)` waits until exactly n bytes are available and returns them, or raises `asyncio.IncompleteReadError` if EOF comes first. The exception carries the bytes it did manage to read in its `partial` attribute and the count it was waiting for in `expected`, which makes the log line for a truncated frame trivially precise. This is the method for length-prefixed protocols: read a fixed-size header with `readexactly`, decode the length from it, then `readexactly` that many body bytes. A short frame raises instead of being handed to your parser as if it were whole. `readexactly` is deliberately **not** subject to the stream's limit, because the length came from the protocol, not from a delimiter search. ## The limit, and which exception it raises A `StreamReader` is created with a limit, 64 KiB by default, which caps how much may be buffered while hunting for a delimiter — otherwise a peer that never sends `\n` would make you buffer forever, which is a denial-of-service primitive as much as a bug. When a single line exceeds it, `readuntil` raises `asyncio.LimitOverrunError` and leaves the data in the buffer, while `readline` catches that and re-raises it as a plain `ValueError` after discarding. Both `asyncio.open_connection` and `asyncio.start_server` take a `limit` keyword argument if your protocol legitimately has longer lines. ## Choosing * You control the protocol and messages can be large or binary → **length prefix** plus `readexactly`. Cheapest to parse, strict about truncation, no escaping needed. * The protocol is line-oriented and text → `readline`, plus an explicit check that the line ended in `\n` if truncation must be detected; or `readuntil` if you would rather it raised. * You are proxying opaque bytes and boundaries do not matter → `read(n)` in a loop until `b""`. ## What interviewers listen for That `read(n)` may return fewer than n bytes; that `readline` swallows truncation while `readexactly` and `readuntil` raise; and that `IncompleteReadError` carries the partial data so the failure can be reported precisely rather than as a bare parse error three layers up.

A scrape ends mid-line and your parser reads a plausible but wrong number. Which read method were you using, and how do you catch it?
`readline`, whose EOF rule is to return the partial line without raising. Either check the return value — a non-empty line that does not end in `\n` means the stream died mid-record — or switch to `readuntil(b"\n")`, which raises `asyncio.IncompleteReadError` on that exact condition and hands you the partial bytes for the log.
Why does asyncio's StreamReader impose a 64 KiB limit on readline, and how do you raise it?
Because a delimiter search must buffer everything it has not matched yet: a peer that never sends the separator would otherwise make you buffer without bound, which is a denial-of-service primitive. Past the limit `readuntil` raises `asyncio.LimitOverrunError` and `readline` raises `ValueError`. Both `asyncio.open_connection` and `asyncio.start_server` accept a `limit` keyword argument to change it.
Can await reader.read(1024) return fewer than 1024 bytes without the stream having ended?
Yes, and it routinely does. `read(n)` returns whatever is buffered up to n, or the first chunk that arrives if the buffer was empty; it never waits to fill the request. Only `b""` means EOF. Code that assumes a full read is the standard source of split-message bugs, since TCP preserves byte order but not message boundaries.

saying these in an interview costs you the question

Assumes read(n) always returns exactly n bytes
Thinks readline raises when the stream ends mid-line
Treats one read() call as exactly one message
Believes readexactly is capped by the 64 KiB limit
Cannot say what IncompleteReadError carries

How do asyncio's StreamReader.read, readline and readexactly differ at end of stream?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Framing is the actual subject

read(n) — no framing at all

readline() — delimiter framing, lenient at EOF

readuntil(sep) — delimiter framing, strict at EOF

readexactly(n) — length framing

The limit, and which exception it raises

Choosing

What interviewers listen for

▾common follow-ups3 questions

▾red flags5 flags

§ context

▾related questions4 questions

▾related topics4 topics

Framing is the actual subject

read(n) — no framing at all

readline() — delimiter framing, lenient at EOF

readuntil(sep) — delimiter framing, strict at EOF

readexactly(n) — length framing

The limit, and which exception it raises

Choosing

What interviewers listen for

answer

go deeper

context