How do max.request.size (producer), message.max.bytes (broker), and replica.fetch.max.bytes interact, and what breaks if they're misaligned?

## The chain a large message must pass through A record produced to Kafka traverses several size gates. Each has its own limit and they must be **consistent** or the record is rejected or stuck somewhere in the pipeline. ### 1. Producer: max.request.size `max.request.size` (default `1048576` = 1 MB) caps the size of a **single produce request**. In practice it bounds the largest individual record the producer will send (a record can't exceed the request that carries it). The producer checks this client-side and throws `RecordTooLargeException` locally before sending if a record exceeds it. Note this is distinct from `batch.size` — `batch.size` is a target for batching efficiency; `max.request.size` is a hard ceiling. ### 2. Broker / topic: message.max.bytes (and max.message.bytes) `message.max.bytes` is the broker-default maximum size of a **record batch** the broker will accept (default ~1 MB; the exact default has shifted across versions). The per-topic override is `max.message.bytes`. If an incoming batch exceeds this, the broker rejects the produce with `MESSAGE_TOO_LARGE` and the client surfaces a `RecordTooLargeException`. Because compression happens before this check on the producer and the broker validates the (possibly compressed) batch size, **compression can let a logically large payload fit under the limit**. ### 3. Replication: replica.fetch.max.bytes Follower replicas fetch batches from the leader to replicate. `replica.fetch.max.bytes` must be **>= the largest batch the broker accepts**. If a leader accepts a 5 MB batch but `replica.fetch.max.bytes` is 1 MB, followers cannot fetch that batch, fall out of the ISR, and replication stalls — historically this could wedge a partition. Modern Kafka mitigates by always allowing at least one batch, but you should still size it correctly. ### 4. Consumer: fetch.max.bytes / max.partition.fetch.bytes Consumers must be able to fetch large batches too. `max.partition.fetch.bytes` and `fetch.max.bytes` need to accommodate them, or consumers can't make progress on partitions holding oversized batches (again, modern clients guarantee at least one batch to avoid total stall). ## What breaks on misalignment | Misalignment | Symptom | |---|---| | producer max.request.size > broker max.message.bytes | Broker rejects: MESSAGE_TOO_LARGE / RecordTooLargeException | | broker max.message.bytes > replica.fetch.max.bytes | Followers can't replicate; ISR shrinks; under-replicated partitions | | broker/topic limit > consumer fetch limits | Consumer can't read oversized records (mitigated by min-one-batch rule) | | producer limit < actual record size | Client-side RecordTooLargeException before send | ## How compression interacts The size checks generally apply to the **compressed batch bytes**. So enabling `compression.type` can shrink a batch below `max.message.bytes`, letting larger logical payloads through. But never rely on compression to dodge a hard limit for variable data — worst-case incompressible payloads can blow past it. ## Operational guidance To support larger messages, bump all four in lockstep: `max.request.size` (producers), `message.max.bytes`/topic `max.message.bytes` (brokers/topics), `replica.fetch.max.bytes` (replication), and `fetch.max.bytes`/`max.partition.fetch.bytes` (consumers). For very large payloads, prefer the **claim-check pattern** (store the blob in object storage, send a reference) instead of pushing limits sky-high.