skip to content

In gRPC, how do grpc-encoding, grpc-accept-encoding and the Compressed-Flag byte negotiate message compression, and what happens when a receiver cannot decode the encoding?

level: middleimportance: nice to knowfreq 24%

answer

  1. what I send versus what I read
  2. one algorithm per direction
  3. the flag decides per message
  4. server rejects unknown algorithm with 12

basics

~20 s

grpc-encoding names the algorithm a sender uses, grpc-accept-encoding lists what it can decode, and each message's Compressed-Flag says whether that message is compressed. A server given an unsupported algorithm fails with UNIMPLEMENTED; a client fails with INTERNAL.

solid answer

~40 s

gRPC compresses individual messages, not the stream. Each direction's headers carry `grpc-encoding`, the one algorithm that sender uses (`identity`, `gzip`, `deflate`, `snappy` or custom), and `grpc-accept-encoding`, the list it can decode. Every message's five-byte prefix starts with a Compressed-Flag: `1` means compressed with the call's `grpc-encoding`, `0` means not, so a sender can skip compression for small or sensitive messages, and each message gets a fresh compression context. Requests and responses negotiate independently; a server sends uncompressed rather than use an algorithm the client's `grpc-accept-encoding` excludes. If the client uses an algorithm the server lacks, the server fails with `UNIMPLEMENTED` and returns `grpc-accept-encoding` with what it does accept; if the server's choice is unknown to the client, the client fails with `INTERNAL`. None of this is HTTP/2 header compression.

code

http · 25 lines
http
HEADERS (flags = END_HEADERS)
:method = POST
:scheme = https
:path = /shop.inventory.v1.StockService/WatchLevels
:authority = api.example.com
te = trailers
content-type = application/grpc+proto
grpc-encoding = gzip
grpc-accept-encoding = gzip,identity

DATA (flags = END_STREAM)
<Compressed-Flag 1><Message-Length><gzip-compressed request>

HEADERS (flags = END_HEADERS)
:status = 200
content-type = application/grpc+proto
grpc-encoding = gzip

DATA
<Compressed-Flag 0><Message-Length><small uncompressed update>
DATA
<Compressed-Flag 1><Message-Length><gzip-compressed large update>

HEADERS (flags = END_STREAM, END_HEADERS)
grpc-status = 0

go deeper

for a junior

Recall the split: grpc-encoding is the algorithm a sender uses, grpc-accept-encoding is what it can decode, and a flag on each message says whether that message is compressed.

for a middle

Explain why the flag makes compression a per-message choice, why each message gets a fresh context, and how requests and responses negotiate independently, including the server's duty to fall back to uncompressed.

for a senior

Show you can diagnose a failed call: UNIMPLEMENTED from the server with a grpc-accept-encoding list versus INTERNAL on the client, and how to tell an encoding rejection from a missing method.

for a principal

Weigh where compression pays: CPU against bandwidth per message, the size-leak risk when secrets share a message with attacker input, and why mixed client fleets make the advertised encoding set a compatibility contract.

## What the two headers and the flag each say gRPC compresses **messages**, not the HTTP/2 stream as a whole. Every message travels in HTTP/2 DATA frames behind a five-byte prefix: a one-byte **Compressed-Flag** and a four-byte big-endian length. Compression is negotiated with two headers and then applied, or not, message by message with that flag. | Signal | Where it travels | What it means | |---|---|---| | `grpc-encoding` | request headers, response headers | the one algorithm this sender uses for any of its messages marked compressed on this call | | `grpc-accept-encoding` | request headers, response headers | a comma-separated list of algorithms this sender can **decode** | | Compressed-Flag | the first byte of every message prefix | `1` = this message is compressed with the call's `grpc-encoding`; `0` = it is not | The values are content codings: `identity` (no transformation), `gzip`, `deflate`, `snappy`, or a custom name. In gRPC, `deflate` means the zlib structure of RFC 1950 around the RFC 1951 algorithm; peers must not send raw deflate data. Two easily confused facts follow from the table: - `grpc-encoding` describes **what I send**; `grpc-accept-encoding` describes **what I can read**. They answer different questions and can name different algorithms. - Headers are sent once per direction, so each direction names at most one algorithm per call. What varies per message is only whether the flag is on. ## Compression is decided per message Because the flag sits on each message, a sender that declared `grpc-encoding: gzip` may still send some messages uncompressed with flag `0`. Typical reasons: - a message is so small that compressing it gains little or makes it larger; - the payload mixes secrets with attacker-influenced data, where compression can leak information through message size (CRIME-style attacks, which the compression specification cites when it motivates per-message control); - the application asked to disable compression, in which case the specification says the next message **must** be sent uncompressed. The specification also says compression contexts are **not** kept across message boundaries: a new context is created for every message. Each compressed message is decodable on its own, at the cost of not reusing a dictionary built from earlier messages on the stream. If `grpc-encoding` is omitted, every Compressed-Flag on that direction must be `0`. A message with flag `1` and no `grpc-encoding` (or `grpc-encoding: identity`) is malformed and fails with **`INTERNAL`**. ## Each direction negotiates on its own Requests and responses are compressed independently. The specification allows asymmetry: a peer **may** answer with a different algorithm from the request's, or with none, whatever the channel and call settings say. The knowledge is also asymmetric: 1. The client lists what it can decode in its request `grpc-accept-encoding`, so the server knows the client's set from the first request. 2. If the server is asked to compress with an algorithm it knows the client does not support, as indicated by the last `grpc-accept-encoding` it received, it **shall** send the message uncompressed. 3. The client, by contrast, does not know in advance what the server supports. It learns that from the server's `grpc-accept-encoding`, most pointedly when a call fails. A peer may choose not to advertise every encoding it supports. But if it receives a message in an encoding it supports without having listed it, it must add that encoding to its response `grpc-accept-encoding`. ## When the receiver cannot decode The outcome depends on which side failed and why: | Situation | Status | Raised by | |---|---|---| | Client compressed with an algorithm the server does not support | `UNIMPLEMENTED` (12) | server | | Server compressed with an algorithm the client does not support | `INTERNAL` (13) | client | | Algorithm supported, but the bytes would not decompress | `INTERNAL` | the receiving side | | Flag `1` without a `grpc-encoding` other than `identity` | `INTERNAL` | the receiving side | On the server-side `UNIMPLEMENTED`, the server includes a `grpc-accept-encoding` header naming the algorithms it does accept, and that list must not contain the one the client used. The description should name both the unsupported encoding and the supported ones. `UNIMPLEMENTED` is also the status for a method the server does not have, so a client reading it should check the server's `grpc-accept-encoding`. The specification closes the ambiguity: if the client's algorithm **is** on that list and the call still returns `UNIMPLEMENTED`, the cause must not be compression. ## Not the same thing as header compression HTTP/2 also compresses, but something else entirely: | | gRPC message compression | HTTP/2 header compression | |---|---|---| | What is compressed | message bytes inside DATA frames | header fields such as `:path` and `grpc-encoding` | | Negotiated by | `grpc-encoding` / `grpc-accept-encoding` | the HTTP/2 connection itself | | Granularity | per message, by the Compressed-Flag | per header block | | How a failure surfaces | `UNIMPLEMENTED` or `INTERNAL` in the call's trailers | an HTTP/2 `COMPRESSION_ERROR`, which gRPC maps to `INTERNAL` | HTTP/2 always encodes a call's header fields with its own scheme; whether the call's messages are compressed is a separate choice, made by the gRPC headers and the flag. Saying "HTTP/2 already compresses gRPC" mixes the two up.

  • A gRPC client receives UNIMPLEMENTED; how does it tell an unsupported encoding from a method the server does not have?
    It reads the server's `grpc-accept-encoding` and the status message. On an encoding rejection the server lists the algorithms it accepts, excluding the one used, and describes both. The specification also rules that if the client's algorithm appears in the server's `grpc-accept-encoding` and the call still returns `UNIMPLEMENTED`, the cause must not be compression, which points at the method or service instead.
  • Why does gRPC let a sender turn compression off for a single message instead of for the whole call?
    Compressing secrets alongside attacker-influenced data can leak those secrets through message sizes, as in CRIME-style attacks, and tiny messages gain nothing from compression. The Compressed-Flag lets a sender exempt exactly those messages while keeping compression for the rest, and the specification requires the next message to go out uncompressed once compression is disabled.
  • What must a gRPC server do when asked to compress a response with an algorithm the client never listed?
    Send the message uncompressed, with Compressed-Flag 0. The server knows the client's set from the last `grpc-accept-encoding` the client sent, and the compression specification says it shall not use an algorithm that list excludes, so the response stays readable instead of failing on the client with `INTERNAL`.

saying these in an interview costs you the question

  • grpc-encoding lists every compression algorithm the peer is able to decode.
  • Once grpc-encoding names gzip, every message on that call must be compressed.
  • A gRPC response must use the same compression algorithm as its request.
  • A server that cannot handle the client's algorithm fails the call with INTERNAL.
  • gRPC message compression is just HTTP/2 header compression applied to the call.
  • One compression context spans the stream, so later messages compress better.