Explain scatter/gather I/O with channels (read into / write from an array of buffers). What problem does it solve?
answer
- scatter = read(ByteBuffer[]) fills buffers in order
- gather = write(ByteBuffer[]) drains buffers in order
- Solves header+body / segmented messages without manual split/join
- Maps to OS readv/writev (vectored I/O)
- Fills/drains sequentially; partial transfers still apply
basics
~20 sScatter/gather lets a channel read into several buffers in one call (scatter) or write from several buffers in one call (gather). It's useful when a message has fixed parts, like a header and a body — you can read straight into separate buffers, or build a message from separate header and body buffers and write it all at once.
solid answer
~50 sScatter/gather I/O uses the ScatteringByteChannel.read(ByteBuffer[]) and GatheringByteChannel.write(ByteBuffer[]) methods. A scattering read fills an array of buffers in order: it fills the first buffer until full, then spills into the next, and so on. A gathering write drains an array of buffers in order, sending all their bytes as one logical write. The problem it solves is structured/segmented data: protocols often have a fixed-size header followed by a variable body. With scatter you read once and the header lands in the header buffer while the body lands in the body buffer — no manual splitting. With gather you keep header and payload in separate buffers (maybe reusing a constant header buffer) and emit them together without concatenating into one big buffer first. Benefits: fewer system calls, no intermediate copying/concatenation, and natural mapping to message layouts. It maps to the OS readv/writev vectored I/O primitives.
code
java · 13 lines// Scattering read: header lands in its own buffer, body in another
ByteBuffer header = ByteBuffer.allocate(20);
ByteBuffer body = ByteBuffer.allocate(4096);
ByteBuffer[] dsts = { header, body };
long total = channel.read(dsts); // fills header fully, then body
header.flip();
body.flip();
// Gathering write: send header + payload as one logical write
ByteBuffer[] srcs = { header, body };
while (header.hasRemaining() || body.hasRemaining()) {
channel.write(srcs); // drains header then body; loop for partials
}go deeper
Knows scatter reads into several buffers and gather writes from several buffers in one call.
Can use read(ByteBuffer[])/write(ByteBuffer[]) for a header+body message and explains the sequential fill order.
Explains the protocol-parsing motivation, the readv/writev mapping, partial-transfer handling, and buffer-reuse benefits.
Decides when vectored I/O genuinely reduces syscalls/copies in a protocol stack versus framing libraries, and sets conventions for buffer pooling and partial-vector retry.
## The problem: data has structure Network and file formats are rarely one flat blob. A typical message is **header + body**, e.g. a fixed 20-byte header (length, type, flags) followed by a variable-length payload. With a single buffer you'd read everything into one `ByteBuffer` and then manually slice it into header and body — fiddly and copy-prone. Conversely, to *send* a message you'd have to concatenate your header buffer and payload buffer into one buffer before writing. Scatter/gather removes both chores. ## The two operations Channels that support this implement: - **ScatteringByteChannel**: `long read(ByteBuffer[] dsts)` — a **scattering read**. It fills the buffers **in array order**: it pours bytes into `dsts[0]` until that buffer is full (no remaining), then continues into `dsts[1]`, and so on. "Scatter" = one incoming stream is *scattered* across multiple destination buffers. - **GatheringByteChannel**: `long write(ByteBuffer[] srcs)` — a **gathering write**. It writes the bytes of `srcs[0]` (from position to limit), then `srcs[1]`, etc., as **one** logical write to the channel. "Gather" = multiple source buffers are *gathered* into one outgoing stream. Both return the total number of bytes transferred across all buffers, and there are overloads taking an `offset`/`length` to use a sub-range of the array. ## Why it maps cleanly to messages Scattering read example — read a fixed header and the rest of the body in one call: ``` ByteBuffer header = ByteBuffer.allocate(20); ByteBuffer body = ByteBuffer.allocate(1024); ByteBuffer[] bufs = { header, body }; channel.read(bufs); // fills header fully, then body header.flip(); body.flip(); ``` The header lands neatly in `header`, overflow goes to `body` — no manual offset math. Gathering write example — emit a constant header plus a payload without concatenating: ``` ByteBuffer[] bufs = { headerBuf, payloadBuf }; channel.write(bufs); // sends header then payload as one write ``` ## The benefits in concrete terms 1. **Fewer system calls**: one `read`/`write` instead of one per segment. The OS exposes this as **vectored I/O** — `readv(2)` and `writev(2)` — which take an array ("iovec") of buffer descriptors. Java's scatter/gather is a thin mapping over these. 2. **No intermediate concatenation/copy**: you don't allocate a big combined buffer just to split or join. 3. **Buffer reuse**: a constant header buffer can be `rewind()`-reused across many writes. 4. **Atomicity of layout**: gathering write submits the whole vector to the kernel together, so the segments are contiguous on the wire (subject to the usual partial-write rules). ## Caveats - **Fill order matters and is sequential**: a scattering read only spills into the next buffer once the current one is *full*. If `dsts[0]` has 20 bytes remaining and you read 25, the first 20 go to `dsts[0]` and 5 to `dsts[1]`. This is exactly why fixed-size header buffers work, but it means a partially-full earlier buffer captures all subsequent bytes up to its limit first. - **Partial transfers still apply**: the total returned can be less than the sum of all `remaining()`s (non-blocking sockets, full send buffers). You may need to retry, tracking which buffers still have remaining. - **flip each buffer** before reading the data out, as always. ## Mental model Scatter = a sorting machine dropping an incoming stream into labeled bins in order; gather = stapling several pages together and mailing them as one envelope.
- In a scattering read with a 20-byte header buffer and a body buffer, what guarantees the header lands separately?The fixed 20-byte capacity of the header buffer. The scattering read fills it completely (its remaining hits 0) before spilling into the body buffer, so the first 20 bytes always land in the header and the rest in the body.
- What OS primitives back Java's scatter/gather?Vectored I/O syscalls readv(2) and writev(2), which take an array of buffer descriptors (iovec) and transfer to/from them in one call.
saying these in an interview costs you the question
- Thinking buffers are filled in parallel or proportionally — it's strictly sequential, filling each before the next
- Forgetting to flip() each buffer after a scattering read
- Assuming the whole vector always transfers in one call (partial transfers happen)
- Confusing scatter (read into many) with gather (write from many)