skip to content

Why use io.ReadFull instead of one Read call on an io.Reader, and what errors does it return?

level: middleimportance: should knowfreq 44%

answer

  1. short reads are legal
  2. an exact count needs a loop
  3. nothing read versus partly read
  4. two different end-of-stream errors
  5. the slice you passed sets the bound

basics

~20 s

One Read on an io.Reader may fill only part of the slice and still return a nil error, so an exact-length read must loop. io.ReadFull is that loop: nil when full, io.EOF if it read nothing, io.ErrUnexpectedEOF if it read part.

solid answer

~50 s

The `io.Reader` contract lets `Read` return fewer bytes than the slice holds with a nil error — on a socket you get whatever has arrived, which for a 4-byte header might be 1 byte. So any code that needs exactly N bytes must loop, and `io.ReadFull(r, buf)` is the standard loop: it reads until `len(buf)` bytes are in hand or it cannot continue. Its three outcomes are the point of the function. A nil error means the slice is completely filled. `io.EOF` means it read zero bytes, which is the clean "the stream ended on a message boundary" case. `io.ErrUnexpectedEOF` means it read some but not all, so the stream is truncated mid-item and that is a protocol error rather than a normal end. `io.ReadAtLeast(r, buf, min)` generalises it, and returns `io.ErrShortBuffer` if you ask for more than the slice can hold.

code

go · 10 lines
go
var hdr [4]byte
n, err := io.ReadFull(conn, hdr[:])
switch {
case err == nil:
	// all 4 bytes are in hdr
case err == io.EOF:
	// n == 0: peer closed cleanly on a message boundary
case err == io.ErrUnexpectedEOF:
	// 0 < n < 4: the header was cut in half, protocol error
}

go deeper

for a junior

Remember that a read can come back partly filled with no error, and that io.ReadFull is the standard way to insist on an exact number of bytes.

for a middle

Be able to state the three outcomes precisely and say which one means a truncated message. Expect to be asked why a file-backed reader hides the bug that a socket exposes.

for a senior

Demonstrate the framed-read discipline end to end: fixed header with io.ReadFull, validate the declared length, then allocate, then io.ReadFull the body, with truncation logged as a protocol error.

for a principal

Decide what the service does on io.ErrUnexpectedEOF as a matter of policy: drop the connection, count it as a peer-side defect, and make sure that signal reaches whoever owns the misbehaving client.

## The short-read rule `io.Reader` has one method: ```go Read(p []byte) (n int, err error) ``` and its documented contract is deliberately weak: a call may read up to `len(p)` bytes and return how many it got. Returning `n < len(p)` with a nil error is completely legal, and on real sources it is normal. A TCP connection returns whatever bytes have arrived in the socket buffer. A pipe returns whatever the writer has flushed. A decompressing reader returns however much fell out of the current block. This is the single most-missed detail in first-time Go I/O code. `conn.Read(buf)` is not "fill buf". It is "give me something, up to this much". A second rule follows from the same contract: **`Read` may return `n > 0` together with `io.EOF`**. The bytes are real and you must process them; checking the error before consuming `n` throws away data. The safe shape is always to handle the `n` bytes first and only then look at the error. ## What io.ReadFull does about it ```go func ReadFull(r Reader, buf []byte) (n int, err error) ``` io.ReadFull loops until it has exactly `len(buf)` bytes, or until the underlying reader stops producing them. It exists so you never hand-roll that loop, and its error vocabulary is the part worth memorising: | outcome | meaning | |---|---| | `n == len(buf)`, `err == nil` | the slice is completely filled | | `n == 0`, `err == io.EOF` | the stream ended before any of these bytes arrived | | `0 < n < len(buf)`, `err == io.ErrUnexpectedEOF` | the stream ended part way through | The distinction between the last two is why this matters for framed protocols. If you are at a message boundary and the peer closes, `io.EOF` is a clean shutdown and your loop should exit quietly. If you are three bytes into a four-byte length prefix and the peer closes, the message was cut in half — that is `io.ErrUnexpectedEOF`, and it deserves a logged protocol error, not a silent exit. Collapsing them into a single "the stream ended" branch turns a corruption bug into a no-op. `io.ErrUnexpectedEOF` is also what well-behaved decoders convert an inner `io.EOF` into when they were mid-item: it is the standard way Go says "the end came where an end is not allowed". ## io.ReadAtLeast, and io.ErrShortBuffer ```go func ReadAtLeast(r Reader, buf []byte, min int) (n int, err error) ``` io.ReadFull is defined in terms of this one: it is `ReadAtLeast(r, buf, len(buf))`. Use io.ReadAtLeast when you need a guaranteed minimum but will happily take more in the same pass — for example when you need at least a header's worth before you can decide how much more to want, and taking whatever else is already buffered saves a syscall. Its error rules are the same shape, with one extra: if `min > len(buf)` the call cannot possibly succeed, and it returns `io.ErrShortBuffer` immediately. ## Why this is a bounded-read question The reason to prefer these helpers over an ad-hoc loop is that they read a **size you chose** — `len(buf)`. You allocated the slice, so the memory cost is under your control, unlike a read-until-end helper where the sender decides. That makes io.ReadFull the natural primitive for a length-prefixed protocol: read a fixed-size header with io.ReadFull, decide from it how big the body is *allowed* to be, and only then allocate and io.ReadFull the body. The decision about the body's size stays yours at every step. ## Common mistakes - Assuming `Read` fills the slice. It does not, and code that assumes it works fine against a file and fails against a socket, which is a horrible way to find the bug. - Discarding `n` when the error is non-nil. Bytes can come back with `io.EOF`. - Treating `io.ErrUnexpectedEOF` as normal termination, which hides truncation. - Using io.ReadFull with a slice sized from untrusted input. io.ReadFull itself is bounded by the slice, but if the peer chose the slice's length you have moved the unbounded allocation one line earlier rather than removing it.

  • Why does io.ReadFull distinguish io.EOF from io.ErrUnexpectedEOF at all?
    Because they mean opposite things to a framed protocol. Zero bytes read means the peer stopped on a clean boundary and your read loop should exit quietly. A partial read means a message was cut in half, which is corruption and should be logged and the connection dropped. Folding both into one branch silently converts truncation into a normal shutdown.
  • What does io.ReadAtLeast add over io.ReadFull?
    It separates the minimum you require from the maximum you will accept in one pass: `io.ReadAtLeast(r, buf, min)` returns as soon as it has `min` bytes but may return more, up to `len(buf)`. That is useful when you need a header before deciding what comes next and would rather take whatever else already arrived. If `min` exceeds `len(buf)` it returns io.ErrShortBuffer without reading.
  • Does using io.ReadFull make a read safe against a hostile peer?
    Only for the bytes it reads, because it is bounded by the slice you passed. If that slice was sized from a length the peer sent, the unbounded allocation has just moved one line earlier, to the make call. io.ReadFull is the safe transfer step; validating the size before allocating is a separate, mandatory step.

saying these in an interview costs you the question

  • Assumes Read always fills the slice it is given
  • Treats io.ErrUnexpectedEOF and io.EOF as the same outcome
  • Checks the error before consuming the n bytes returned with it
  • Writes a manual read loop but drops the returned byte count
  • Thinks io.ReadFull blocks until the whole slice is available regardless of the peer closing