skip to content

How can `Content-Encoding: gzip` on a Server-Sent Events response defeat incremental delivery, and which directive pushes back?

level: middleimportance: should knowfreq 42%

answer

  1. the coding may not be yours
  2. input accumulates before output appears
  3. a block is many small events wide
  4. no-transform is the standard signal
  5. it stops transformation, not accumulation

basics

~20 s

A compressor that accumulates input until it has a block worth of bytes emits nothing in the meantime, so events disappear into it and surface in groups. Cache-Control: no-transform is the standard directive telling an intermediary not to change the body's content coding.

solid answer

~40 s

Compression is not inherently hostile to streaming -- a compressor can flush after every event, at a real cost in ratio -- but the default in most stacks is to fill a block before emitting output, and a block is many events wide on a feed of short ones. A compressing intermediary is also, by construction, a buffering one. The sharper version of the problem is that the coding may not be yours: a transforming intermediary can apply `Content-Encoding: gzip` to a response whose origin never set it, on the strength of the request's `Accept-Encoding`. `Cache-Control: no-transform` is the standard signal that an intermediary must not change the content coding or media type of the payload, and it belongs on any streamed response.

code

http · 7 lines
http
HTTP/1.1 200 OK
Content-Type: text/event-stream
Cache-Control: no-cache, no-transform

data: speed=412

data: jam=0

go deeper

for a junior

Recall that compressing a live feed can delay it, and that the standard directive asking intermediaries to leave a body untouched is no-transform.

for a middle

Explain that a compressor accumulates input before emitting output unless it is flushed, and that an intermediary can add a coding the origin never set.

for a senior

Diagnose it end to end: read the header fields as they arrive rather than as configured, separate transformation from accumulation, and decide whether any coding belongs on the feed at all.

for a principal

The call is whether compression policy is set per media type across the fleet or left to each edge tier, given that a transforming hop can override the origin's intent on a response nobody thought to exempt.

## Two places the coding can come from When a bottling-line feed arrives compressed, there are two candidate sources and they have different fixes: 1. **The origin applied it.** A server framework that compresses responses by content type may compress `text/event-stream` along with everything else, because a text media type looks like an obvious win. 2. **An intermediary applied it.** A transforming intermediary is entitled to apply a content coding the origin never set, choosing it from the codings the client advertised in the request's `Accept-Encoding` field. `Content-Encoding` is the response field naming what was applied; `Accept-Encoding` is the request field naming what the client will accept. Mixing those two up is the most common slip in this answer. The second case is the one that surprises people, because the origin's own configuration shows no compression anywhere and the response on the wire is compressed regardless. ## Why compression eats a live feed A compressor works over a window of input. To choose good encodings it wants input, and the cheapest implementation strategy is to accumulate until it has a block worth of bytes and only then emit compressed output. On a feed of short events -- a line number, a speed, a jam flag -- a block is many events wide, so the first twenty events produce **no output at all**, then all twenty appear together. The important nuance, and the one that keeps this claim honest: a compressor **can** be told to flush at a chosen point, emitting whatever it has so far and paying for it in ratio, and stacks that support streaming compression do exactly that. So compression does not necessarily destroy incremental delivery. What destroys it is a compressor whose flush policy is chosen for whole documents applied to a response that is not one. There is a second cost even when flushing is correct: a compressing intermediary has to read, decode and re-encode the body, which means it is handling the body rather than passing it through -- so it is a buffering intermediary too, with all of the behaviour that implies. ## The lever, and its limits `Cache-Control: no-transform` is the standard directive for this. It tells intermediaries on the path that they must not change the payload -- specifically not its content coding or its media type. Put it on every streamed response; it costs one token and it is the only standard way to say "leave this body exactly as it is". Its limits are worth stating plainly: - It addresses **transformation**, not accumulation. An intermediary that honours `no-transform` and still buffers the untouched body produces the same clumping. - It is a directive, and an intermediary that ignores directives ignores this one. - It does nothing about compression the **origin** applied, which is yours to turn off. ## Deciding what the origin should do | Setting | Effect on a live feed | When it is right | |---|---|---| | No content coding on the stream | Each event is forwarded as written | The default choice; event payloads are small and the ratio is not worth the risk | | Streaming compression, flushed per event | Incremental delivery preserved, ratio reduced | Large event payloads over constrained links, when the stack genuinely supports per-event flushing | | Block-buffered compression | Incremental delivery destroyed | Never on a response that is meant to arrive as it is produced | For most feeds the first row wins. The events are short, the saving is small, and the failure mode is invisible until it reaches production. ## How to diagnose it 1. Read the response header fields as they arrive at the receiver, not as the origin configured them. If `Content-Encoding` is present and the origin never set it, a transforming intermediary added it. 2. Request the same path with no compression offered in the request. If the feed becomes smooth, the coding was the cause rather than a separate buffering hop. 3. If it is still clumpy without any coding, you are looking at plain response buffering instead, and compression was a red herring. ## Why interviewers like this one It catches two habits at once. The first is assuming that everything on the wire was put there by your code -- the response you configured is not necessarily the response that arrives. The second is treating compression as a pure win, which it is on documents and is not on anything that must arrive as it is produced. A candidate who reaches for `no-transform` and then immediately says "but it does not stop buffering" has the whole shape of this leaf.

  • The origin never sets a content coding, yet the response arrives compressed. How?
    A transforming intermediary applied one. It may choose a coding from those the client advertised in the request's `Accept-Encoding` field and apply it to a response the origin sent uncompressed. `Cache-Control: no-transform` is the standard way to tell it not to, and reading the header fields as they arrive at the receiver rather than as configured at the origin is how you catch it.
  • If you must compress a large event payload, how do you keep the feed incremental?
    Use a compressor that flushes at a boundary you choose - typically after each dispatched event - so whatever has been encoded so far is emitted rather than held for the next block. You pay for it in compression ratio, because each flush ends a block early, and the trade is only worth making when the payloads are large enough for the ratio to matter.

saying these in an interview costs you the question

  • Says compression and streaming are simply incompatible
  • Assumes only the origin can apply a content coding
  • Confuses Accept-Encoding on the request with Content-Encoding on the response
  • Believes no-transform also prevents an intermediary buffering the body
  • Claims a compressed body cannot be sent incrementally at all