Why must an io.Writer implementation not retain or modify the caller's slice p after Write returns?
answer
- who owns the slice after the call?
- io.Copy refills one scratch slice
- the doc forbids retaining p
- an async handoff needs a copy first
basics
~20 sThe io package requires that Write neither keep the caller's slice p after returning nor modify its contents. Callers reuse one slice for an entire stream, so a retained slice is overwritten underneath the implementation, corrupting its data.
solid answer
~40 s`io.Writer`'s documented contract says an implementation must not retain `p` and must not modify the slice data, even temporarily. That rule exists because callers reuse buffers: `io.Copy` reads into one scratch slice, hands it to `Write`, then immediately reads the next chunk into the same backing array. If your `Write` pushes `p` onto a channel for a background goroutine to flush later, that goroutine reads bytes that have already been replaced -- so the output is a scrambled mix of chunks, and because two goroutines touch the same array with no synchronisation it is a genuine data race that `go test -race` will report. The fix is to copy before you hand off: allocate or take a slice you own, `copy(b, p)`, then queue `b`. `io.Reader.Read` carries the same rule for the same reason.
code
go · 16 linestype badWriter struct{ ch chan []byte }
// WRONG: retains p; the caller refills that array on the next read.
func (w *badWriter) Write(p []byte) (int, error) {
w.ch <- p
return len(p), nil
}
type goodWriter struct{ ch chan []byte }
func (w *goodWriter) Write(p []byte) (int, error) {
b := make([]byte, len(p))
copy(b, p)
w.ch <- b
return len(p), nil
}go deeper
Remember the rule as written: an implementation of Write may not keep the slice it is given, and may not change its contents. If you need the bytes later, copy them first.
Be ready to explain the mechanism -- callers reuse one buffer across the whole stream, so a retained slice is silently refilled with the next chunk, and a background reader of it is also a data race.
Show how you would find this in a running system: corruption that only appears on large payloads, reproduced under the race detector, and a review habit of looking for p escaping the method body.
Decide and document the ownership rule for every byte slice your package's exported API accepts or hands out. Matching the io convention costs a copy and saves every consuming team from a class of unreproducible corruption.
## The clause The `io` package documents a duty on the *implementer* of `Read` and `Write`, not on the caller. For `Write` it reads, in substance: `Write` must not modify the slice data, even temporarily, and implementations must not retain `p`. `Read` carries the matching sentence: implementations must not retain `p`. "Retain" means keep a reference that outlives the call -- storing it in a struct field, appending it to a slice of pending chunks, sending it on a channel, or capturing it in a closure that runs later. Once `Write` returns, the implementation must be finished with those bytes. ## Why the rule exists The whole design of `io` is built on the caller owning and reusing the memory. `io.Copy` allocates a single scratch slice and loops: read into it, write from it, read into it again. The second read overwrites the same backing array the previous `Write` was given. An implementation that kept that slice is now looking at the *next* chunk of the stream, or at a half-overwritten mixture of two chunks. That produces a bug with a distinctive shape. Small inputs work, because everything fits in one iteration and there is never a second read to overwrite anything. Large inputs corrupt -- and often corrupt differently on every run, because the corruption depends on how the reader chunked the data and on goroutine timing. Nothing panics; the destination simply contains bytes that never appeared in the source, or the same 32 KiB twice. If the retained slice is read from a different goroutine than the one refilling it, this is not merely a logic bug but an unsynchronised concurrent access to the same memory. `go build -race` / `go test -race` will report it as a data race, with both stacks -- which is usually the fastest way to find it, because the corrupted output alone rarely points at the guilty `Write`. ## The "even temporarily" part The `Write` contract goes further than "do not keep it": it forbids modifying `p` at all, including a change you undo before returning. A tempting trick when framing a protocol is to write a length prefix into the front of the caller's slice, send it, then restore the original bytes. That is forbidden, and for good reason -- the caller may be writing the same slice to several destinations, or may be sharing it read-only with another goroutine that is entitled to assume nobody mutates it. The caller's data is input, not scratch space. ## Doing it correctly If your implementation genuinely needs the bytes after `Write` returns -- an asynchronous flusher, a retry queue, an in-memory accumulator -- copy them into memory you own: ```go b := make([]byte, len(p)) copy(b, p) ``` or, equivalently, `b := append([]byte(nil), p...)`. If allocation per write is too expensive, take the destination slice from a pool you manage and return it when the flush completes. The cost of that copy is precisely the cost of the guarantee you are making; there is no way to keep the caller's memory and stay inside the contract. Note also the appending trap: `w.pending = append(w.pending, p...)` copies the bytes and is fine, while `w.chunks = append(w.chunks, p)` stores the slice header itself and is a violation. The two lines differ by three characters. ## The mirror image on the reader side A `Read` implementation must not keep `p` either. A reader that stashes the caller's slice and fills it later -- from a background goroutine, or on the next call -- writes into memory the caller may have already handed elsewhere, reused for another stream, or shrunk. The caller is entitled to do anything it likes with `p` the moment `Read` returns. The symmetric duty on the *caller* side is not to look at `p[n:]`. Those bytes are whatever was left over from previous use; the implementation makes no promise about them. ## Reviewing for it When you review an `io.Writer` or `io.Reader` implementation that another team will use, this is the first thing to check, because the compiler cannot. Look for `p` escaping the method body: stored in a field, sent on a channel, captured by a goroutine, appended as a whole slice, or passed to something that outlives the call. `go build -gcflags=-m` reporting that `p` escapes to the heap is a useful hint that something is holding on to it, though not proof on its own. And document your own types' behaviour explicitly when you accept a slice outside the `io` interfaces. The `io` rule is a convention people rely on precisely because it is uniform; a type that quietly breaks it will be misused by everyone who assumes the standard shape.
- Is w.pending = append(w.pending, p...) also a violation?No. The `...` form copies the bytes into a slice the implementation owns, so nothing of the caller's memory is retained. The violation is `w.chunks = append(w.chunks, p)`, which stores the caller's slice header itself. The two lines look almost identical and behave completely differently.
- What symptom would make you suspect a retained buffer in production?Output that is correct for small payloads and corrupt for large ones, with the corruption looking like duplicated or interleaved chunks rather than random bytes -- because the damage only appears once the caller's loop runs a second iteration. Re-running under `-race` usually names the two goroutines involved immediately.
- Does the caller have any matching obligation?Yes: after `Read` returns n, the caller may only trust `p[:n]`. The remaining bytes are stale leftovers, and reading them is the mirror-image bug. Beyond that the caller is free to reuse, resize or discard the slice the moment the call returns.
It is like being handed a whiteboard to read from: copy down what you need before you walk away, because the next person is going to wipe it and write something else.
saying these in an interview costs you the question
- Sends the caller's slice to a goroutine without copying
- Stores p in a struct field for later flushing
- Says modifying p is fine if you restore it before returning
- Assumes each Write receives a freshly allocated slice
- Confuses append(dst, p...) with append(dst, p)