skip to content

Why can an io.Reader's Read return n > 0 bytes and io.EOF in the same call?

level: middleimportance: must knowfreq 58%

answer

  1. two return values, not two alternatives
  2. the end can arrive with the data
  3. bytes first, error second
  4. a wrong loop still passes against a file
  5. testing/iotest ships the adversarial reader

basics

~20 s

The io.Reader contract lets a reader that hits the end of its data while filling your buffer report both at once. Process the n bytes returned before you inspect the error, or the last chunk of the stream is silently dropped.

solid answer

~50 s

The `io.Reader` contract explicitly permits a reader that reaches the end of input after successfully delivering `n > 0` bytes to return the count and `io.EOF` from the same call — or to return `(n, nil)` now and `(0, io.EOF)` on the next call. Both are legal, so a caller has to handle both. That gives one hard rule: always consume the `n` bytes first, then look at the error. Code shaped `if err != nil { break }` before touching `buf[:n]` loses the final chunk of every stream whose reader takes the combined form, and the bug is invisible against readers that take the other form. Files usually report the end separately, while wrappers over the network or over compression often combine them, which is why the bug reaches production. `testing/iotest.DataErrReader` exists to force the combined shape in a test.

code

go · 12 lines
go
for {
	n, err := r.Read(buf)
	if n > 0 {
		consume(buf[:n]) // valid data even when err is io.EOF
	}
	if err == io.EOF {
		return nil
	}
	if err != nil {
		return err
	}
}

go deeper

for a junior

Remember the ordering: use the n bytes Read gave you before you look at the error, and always slice the buffer as buf[:n] rather than using all of it.

for a middle

Be able to state both legal shapes of the ending — bytes with io.EOF in one call, or bytes now and io.EOF on the next — and explain why implementations are allowed the choice.

for a senior

Show how you would prove a loop correct rather than assume it: run it over the adversarial readers in testing/iotest so the combined-return case is exercised, since a plain file will never expose the bug.

for a principal

Own the argument for not hand-writing read loops at all: every framing layer that rolls its own repeats this bug, so the reusable helper or the standard library call is the cheaper long-term position.

## What the contract actually promises The documented behaviour of `Read(p []byte) (n int, err error)` is deliberately loose in one specific way. When a reader hits an error or the end of its input *after* successfully filling `n > 0` bytes, it is allowed to do either of two things: 1. return `(n, io.EOF)` — the data and the ending in the same call; or 2. return `(n, nil)` now and report `(0, io.EOF)` on the following call. Both are conforming. The contract also tells callers what to do about it in one line: *process the n > 0 bytes returned before considering the error*. Everything else in this topic follows from that sentence. ## Why the loose contract exists Go's readers are thin wrappers over whatever is underneath — a file descriptor, a socket, a decompressor, a byte slice in memory. Some of those learn that the source is exhausted as part of the very syscall that returned the last bytes; forcing them to hide that knowledge and return it on a later call would mean an extra bookkeeping state in every implementation, and often an extra syscall. Allowing the combined form keeps implementations cheap. The cost is pushed onto callers, and it is a small cost — provided callers obey the ordering rule. ## The bug this produces ``` // WRONG: drops the final chunk for { n, err := r.Read(buf) if err != nil { break } consume(buf[:n]) } ``` Against `os.File`, which normally reports the end on a separate call, this loop looks perfect and passes every test. Point it at a reader that combines the two — many network and decompression wrappers do — and the last few kilobytes vanish, with no error anywhere. The symptom is a file that is *almost* right: a truncated last record, a JSON document missing its closing brace, a checksum that does not match. The correct shape puts the data first: ``` for { n, err := r.Read(buf) if n > 0 { consume(buf[:n]) } if err == io.EOF { return nil } if err != nil { return err } } ``` Note that the rule is not special to `io.EOF`. If a reader returns `(n, someRealError)`, those `n` bytes are still real data that the source produced; whether you keep them is your decision, but the loop must at least be aware of them. ## The other end of the range The contract also says something about the empty case: a return of `(0, nil)` is *not* a report that the stream ended. It means nothing happened, and a caller should treat it as a no-op and read again. Implementations are discouraged from producing it, except when `len(p) == 0`. So `(0, nil)` never means "done", and only `io.EOF` does. ## Why you rarely see this yourself Most application code does not write raw read loops. `io.Copy`, `io.ReadAll`, `bufio.Reader` and `bufio.Scanner` already implement the rule correctly, which is a strong argument for using them instead of hand-rolling. The moment you *do* hand-roll — a framing layer, a chunked protocol, a progress-reporting wrapper — the rule becomes yours to honour. ## Testing for it The standard library ships adversarial readers precisely for this, in `testing/iotest`: - `iotest.DataErrReader(r)` wraps a reader so the final error arrives *with* the final data rather than on a separate call — exactly the shape that breaks the wrong loop. - `iotest.OneByteReader(r)` returns one byte per call, exposing loops that assume the buffer is filled. - `iotest.HalfReader(r)` returns half of the requested bytes per call. - `iotest.ErrReader(err)` returns `(0, err)` immediately, for the failure path. Running the same read loop over the same input through each of these, and asserting the same output every time, is what actually pins the behaviour down. A single test against a `strings.Reader` proves very little. ## The one-line summary to say out loud The byte count and the error are independent return values, not alternatives. Read the count first; interpret the error second.

  • What does a Read returning (0, nil) mean?
    Nothing happened. It is explicitly not an end-of-stream report, and callers should treat it as a no-op and read again. Implementations are discouraged from returning it except when the caller passed a zero-length buffer, because a caller that loops on it burns CPU for no progress.
  • Why does the same read loop pass its tests against a file and lose data over a network stream?
    Files typically report the end on a separate call, so a loop that checks the error before consuming the bytes never sees a combined return. Wrappers over sockets and decompressors often learn the source is exhausted in the same operation that produced the final bytes, so they return the count and io.EOF together — and the loop drops that last chunk.
  • Does the ordering rule apply to errors other than io.EOF?
    Yes. The contract is about n > 0 generally: if a Read returns bytes alongside any error, those bytes were genuinely produced by the source. Whether you keep them is a judgment call for the format, but the loop must not pretend they never arrived.

saying these in an interview costs you the question

  • Checking err before touching the n bytes returned
  • Assuming io.EOF always arrives on a call that returned zero bytes
  • Reading buf up to its full length instead of buf[:n]
  • Treating a (0, nil) return as the end of the stream
  • Testing a read loop only against a strings.Reader or a file