skip to content

In a Go pipeline handing `[]byte` pixel buffers between stages, what breaks if a stage reuses its scratch buffer?

level: seniorimportance: nice to knowfreq 30%

answer

  1. the send copies three words, not the pixels
  2. two stages, one backing array
  3. a send hands over ownership
  4. reuse only after it comes back
  5. a clean -race run proves only that run

basics

~20 s

A channel send copies only the slice header, so both stages point at the same backing array. Overwriting it for the next frame corrupts the frame already in flight, so treat a send as handing over ownership of those bytes.

solid answer

~50 s

Sending a `[]byte` over a channel copies the three-word header — pointer, length, capacity — not the bytes. After `out <- buf[:n]` the decode stage and the resize stage hold the same backing array, so the moment decode reads the next photo into `buf` it is rewriting pixels resize has not finished with. Usually that is a genuine data race and `-race` can catch it, but only in a run where the two accesses actually overlap, so a small test passes clean while production produces corrupt thumbnails. The discipline is that a send transfers ownership: once a value has gone down the channel, the sender must not touch anything reachable from it. Either allocate a fresh buffer per frame, or make recycling explicit with a second channel on which the downstream stage returns buffers it has finished with. The same caution applies to a struct carrying a slice, map or pointer field.

code

go · 5 lines
go
buf := make([]byte, 4<<20)
for _, path := range paths {
	n := readFrame(buf, path) // overwrites bytes the resize stage still holds
	out <- buf[:n]
}

go deeper

for a junior

Know that a slice variable is a small header pointing at a separate array, and that copying the header — by assignment, by argument passing, or over a channel — leaves both copies pointing at the same bytes.

for a middle

Explain exactly what a channel send copies for a slice, for a struct and for a pointer, and why two stages end up sharing state even though they pass values through a channel rather than sharing a variable.

for a senior

Show the ownership discipline you would enforce in review: a sent value belongs to the receiver, any recycling is explicit through a return channel, and a clean race-detector run is evidence about one execution only.

for a principal

Weigh the cost side: buffer reuse buys allocation savings that only a measurement justifies, and it trades a pipeline that is correct by construction for one whose correctness rests on a convention every future contributor must learn.

## What a channel send actually copies Go channels copy values. That is the whole guarantee, and the trap is that for a slice the *value* is not the data. A `[]byte` is a three-word header: a pointer to a backing array, a length, and a capacity. Sending it copies those three words. The receiver gets its own header — reslicing on one side will not change the other side's length — but both headers point at the same array of bytes. So `out <- buf[:n]` does not hand over a photo. It hands over a *view* of a photo that the sender can still write through. ## The corruption timeline With one scratch buffer in the decode stage: 1. decode reads frame 1 into `buf` and sends `buf[:n]`; 2. resize receives that header and starts reading pixels out of the array; 3. decode loops, opens the next file, and reads frame 2 **into the same array**; 4. resize is now resizing a blend of frame 1 and frame 2. Nothing fails loudly. There is no bounds violation, no nil dereference, no failed send. The only symptom is wrong output: a thumbnail with a band of the wrong image in it, usually only under load, usually only in production. ## Is it a data race? Usually yes. The channel send/receive pair establishes a happens-before edge for the handover itself, but it says nothing about the sender's *later* writes relative to the receiver's reads. Those two accesses are unsynchronised, and the race detector can report them. The important qualifier is that the detector is a run-time observer, not a static check: it reports races that actually occur in the execution you exercise. A unit test with three small files, where each stage happens to finish before the next write lands, passes clean and proves nothing about a production run with large frames and every core busy. "`-race` was green" is evidence about one schedule, and treating it as proof of safety is the misconception this question exists to catch. There is also a variant with no race at all and still-wrong data — a buffered channel that lets decode run several frames ahead and overwrite an entry that has been received but not yet read. Ordering is not the same thing as ownership. ## The rule: a send transfers ownership State it as a rule about values, not about channels: **once a value has been sent, the sender must not touch anything reachable from it.** For an `int`, a `string` or an array field there is nothing reachable, and the copy really is complete. For a slice, a map, a pointer or a channel field, the header or pointer is copied and the target is shared, so a struct like ```go type Frame struct { Path string Pix []byte // shared with the receiver Width int // genuinely copied } ``` is half-copied: `Width` is independent, `Pix` is not. ## Two ways to hold the rule **Allocate per item.** `buf := make([]byte, n)` inside the loop. Each frame owns its bytes, nobody shares anything, and the collector cleans up as frames fall out of the chain. This is the default, and it is the right default: it is correct by construction and needs no comment. **Recycle explicitly.** Add a second channel of free buffers flowing the other way. The decode stage receives a buffer from it, fills it, and sends it downstream; the last stage that touches the pixels sends the buffer back on the free channel when it is finished. A buffer is reused only after its consumer has explicitly given it up, so ownership is still single-writer, and the free channel's capacity caps how many buffers exist at once. It is more machinery, and the machinery is the point: the handover is now visible in the code rather than assumed. ## When is recycling worth it? Only when a measurement says so — large per-frame buffers churning the heap, visible as allocation volume in a heap profile and as bytes per operation in a benchmark run with `-benchmem`. Reuse as a habit, applied before measuring, buys nothing and costs correctness. And when you do adopt it, weigh the ongoing cost honestly: you have converted a pipeline that was safe by construction into one whose correctness depends on a convention that every future contributor has to learn and every reviewer has to check.

  • Does sending a struct by value protect the fields inside it?
    Only the flat ones. The struct is copied on send, so an int, a bool or an array field is genuinely independent. A slice, map, pointer or channel field copies its header or pointer, so both stages still reach the same underlying data, and mutating it upstream is exactly the same bug wearing a struct.
  • When is reusing buffers across pipeline stages worth the risk at all?
    When a measurement says allocation is the cost: large per-frame buffers churning the heap, visible as allocation volume in a heap profile and as bytes per operation in a benchmark run with `-benchmem`. Then make the handover explicit with a free-buffer channel flowing back upstream. Reuse adopted as a habit, before measuring, only buys risk.
  • Why does the pipeline produce corrupt output with no panic and no error?
    Nothing in Go checks who may write to a backing array. The bytes simply change under the reader: no bounds violation, no nil dereference, no failed send. The only symptom is wrong output, which is why the bug survives a small test suite and first appears as a corrupted image in production.

saying these in an interview costs you the question

  • Believes a channel send deep-copies the slice's bytes
  • Reuses one scratch buffer because the sends look sequential
  • Says a clean -race run proves the reuse is safe
  • Copies into the same array before each send and calls it fixed
  • Assumes copying a struct also copies its slice and map fields