skip to content

Using the browser Fetch API, how do you consume a response incrementally instead of awaiting response.text(), and what problem does piping response.body through a TextDecoderStream solve?

level: middleimportance: should knowfreq 45%

answer

  1. the body is a stream, not a value
  2. getReader, then loop on read
  3. chunks are bytes, not characters
  4. a character can straddle two chunks
  5. pipeThrough keeps decoder state

basics

~20 s

response.body is a ReadableStream of byte chunks: call getReader() and loop on read() until done. Piping it through TextDecoderStream turns those bytes into text correctly even when a multi-byte character is split across two chunks.

solid answer

~50 s

`fetch()` resolves as soon as the headers arrive, so `response.body` hands you the rest as a `ReadableStream` of `Uint8Array` chunks. Call `response.body.getReader()` and loop on `await reader.read()`, which yields `{ value, done }`, until `done` is true — that lets you show progress or render tokens as they land instead of waiting for the whole payload the way `response.text()` does. The chunks are bytes, and their boundaries are arbitrary: a multi-byte UTF-8 character can straddle two of them, so decoding each chunk with a fresh `TextDecoder` produces replacement characters at the seams. `response.body.pipeThrough(new TextDecoderStream())` fixes that — you get a stream of strings, and any trailing partial character is held back until the next chunk completes it. Framing is still your job: buffer the text and split on your own delimiter, because a chunk is not a line and not a JSON object.

code

javascript · 18 lines
javascript
async function streamLines(url, onLine) {
  const res = await fetch(url);
  if (!res.ok) throw new Error('HTTP ' + res.status);
  if (!res.body) return;

  const reader = res.body.pipeThrough(new TextDecoderStream()).getReader();
  let buffer = '';

  for (;;) {
    const { value, done } = await reader.read();
    if (done) break;
    buffer += value;
    const parts = buffer.split('\n');
    buffer = parts.pop();
    for (const line of parts) if (line) onLine(line);
  }
  if (buffer) onLine(buffer);
}

go deeper

for a junior

Know that fetch() exposes the body as a stream on response.body, and that response.text() or response.json() waits for the whole thing. Be able to say which one a progress UI needs.

for a middle

Explain the read loop — getReader(), the { value, done } pair, the lock a reader holds — and why bytes must go through a streaming decoder rather than one TextDecoder call per chunk.

for a senior

Show that you treat chunk boundaries as meaningless: describe the buffer-and-split framing you keep on top of the stream, and how you handle a read that rejects part-way through a transfer.

for a principal

Own the tradeoff. Streaming buys first-byte latency and bounded memory and costs framing, retry and testing complexity; be ready to say when a plain buffered response is the right call for a team and where the streaming code should live.

## Two ways to take a body A `fetch()` promise resolves as soon as the response *headers* have been received; the body may still be arriving over the network. The convenience methods — `response.text()`, `response.json()`, `response.blob()` — each buffer the whole body and resolve only when the last byte lands. For a 40 KB JSON document that is exactly what you want. For a large export, a progress UI, or a token-by-token model response it is the wrong shape: nothing can be shown until everything has arrived, and the entire payload sits in memory at once. The alternative is `response.body`, a `ReadableStream` whose chunks are `Uint8Array` values. It is `null` for a response that has no body at all, so a defensive read checks for it. ## The read loop ```js const res = await fetch('/export.ndjson'); if (!res.ok) throw new Error(`HTTP ${res.status}`); if (!res.body) throw new Error('no body to stream'); const reader = res.body.getReader(); for (;;) { const { value, done } = await reader.read(); if (done) break; // value is undefined on the final read console.log(value.byteLength); // value is a Uint8Array } ``` `getReader()` **locks** the stream: while a reader is attached nothing else may read it, and a second `getReader()` throws. `reader.releaseLock()` gives the stream back; `reader.cancel()` discards the rest. `read()` returns a promise for `{ value, done }`, and the terminating read has `done: true` with `value` undefined — a loop that reads `value` before checking `done` is a bug. ## Chunks are bytes, and they are not messages Two independent mistakes live here. First, **chunk boundaries are arbitrary**. They are decided by network packetisation, decompression and the browser's own buffering, not by the server's writes. You may get one chunk for a whole response, or a chunk that ends halfway through a line. Never assume one chunk equals one record. Keep a buffer, append each decoded chunk, split on your delimiter, and carry the trailing fragment forward: ```js let buffer = ''; buffer += text; const parts = buffer.split('\n'); buffer = parts.pop(); // last element may be a partial line ``` Second, **a chunk is bytes, not characters**. UTF-8 encodes most non-ASCII characters in two to four bytes. If a chunk boundary falls between the two bytes of `é`, then `new TextDecoder().decode(chunk)` on each chunk independently decodes each half to U+FFFD, the replacement character, and the text is quietly corrupted. Nothing throws; you just get `��` in the output. ## The streaming decoder The platform gives you two forms of the fix. The stream form is a transform: ```js const textStream = res.body.pipeThrough(new TextDecoderStream()); const reader = textStream.getReader(); // value is now a string ``` `pipeThrough` takes a `TransformStream`, pipes the stream into its writable side, and returns its readable side — so it also locks the original. `TextDecoderStream` keeps decoder state across chunks: a trailing incomplete character is buffered and emitted once the bytes that finish it arrive. The manual form is the same state, held by hand: ```js const decoder = new TextDecoder(); // utf-8 by default text += decoder.decode(value, { stream: true }); // hold back partials // after the loop, flush anything left: text += decoder.decode(); ``` Because `pipeThrough` accepts any `TransformStream`, you can build a pipeline: decompress, decode, then a transform of your own that emits whole lines, so the read loop deals only in records. ## Async iteration Some engines let you write `for await (const chunk of res.body)`, which is much nicer than the reader loop. Support has been uneven — Safari in particular lacked it long after Chrome and Firefox shipped it — so `getReader()` remains the portable form unless you have verified your targets. ## When not to stream If the endpoint returns a modest JSON document that you are going to parse in one go anyway, streaming buys nothing and costs you framing code, error handling and tests. Stream when time-to-first-render matters, when the payload is large enough that buffering it is a memory problem, or when the response is genuinely an open-ended sequence of records.

  • How would you drive a progress bar from a streamed response, and what can stop you?
    Sum `value.byteLength` as chunks arrive and compare it against the `Content-Length` response header. It breaks down in two common cases: a chunked response often carries no `Content-Length` at all, and when the response was compressed on the wire, `Content-Length` describes the compressed size while the bytes you count are decompressed. In both cases you can honestly show bytes received, but not a percentage.
  • Can you put your own transform in front of the read loop?
    Yes. `pipeThrough` accepts any `TransformStream`, so you can chain: `response.body.pipeThrough(new DecompressionStream('gzip')).pipeThrough(new TextDecoderStream())`. Each `pipeThrough` locks the stream it consumes and returns the readable side of the transform, so you read from the end of the chain. Writing a `TransformStream` with a `transform(chunk, controller)` function lets you move line-splitting into the pipeline instead of the loop.
  • When would you deliberately not stream a response?
    When the payload is small and must be parsed as a whole anyway — a normal JSON API response. `response.json()` is one line, is hard to get wrong, and gives the parser the complete document. Streaming adds framing code, partial-failure states and tests, so it earns its place only when first-byte latency, bounded memory, or an open-ended record sequence actually matters.

saying these in an interview costs you the question

  • Thinks await response.text() delivers data progressively
  • Decodes each chunk with a separate TextDecoder call
  • Assumes one chunk is one line or record
  • Believes chunk boundaries mirror the server's writes
  • Reads value before checking the done flag

context