skip to content

How can max.in.flight.requests.per.connection cause message reordering with retries, and how do you prevent it while keeping throughput?

level: seniorimportance: must knowfreq 65%

answer

  1. default in-flight = 5
  2. B1 retried after B2 succeeds -> reorder
  3. old fix max.in.flight=1 (slow)
  4. enable.idempotence: PID + sequence numbers
  5. ordered + dedup up to 5 in-flight
  6. OutOfOrderSequenceException on broken sequence

basics

~20 s

If more than one request is in flight per connection and an earlier batch is retried while a later one already succeeded, the retried batch lands after it, reordering messages within a partition. Enabling idempotence (enable.idempotence=true) prevents reordering even with up to 5 in-flight requests.

solid answer

~40 s

max.in.flight.requests.per.connection controls how many unacknowledged ProduceRequests the producer allows per broker connection (default 5). With retries and >1 in flight, ordering within a partition can break: batch B1 fails and is retried while B2 (sent right after) already succeeded, so on retry B1's records are appended after B2's. The classic safe-but-slow fix was max.in.flight=1. The modern fix is enable.idempotence=true: each batch carries a producer ID and monotonic sequence number, and the broker rejects out-of-order or duplicate sequences, so the producer can keep up to 5 in flight and still guarantee per-partition ordering and exactly-once-per-producer-session dedup. If sequences arrive out of order the broker returns OutOfOrderSequenceException, and the idempotent producer re-sends in the correct order.

go deeper

for a junior

Know that retries plus multiple in-flight requests can reorder messages, and idempotence fixes it.

for a middle

Explain the B1-retried-after-B2 scenario and that max.in.flight=1 is the old fix, idempotence the new one.

for a senior

Describe the PID + sequence-number mechanism and the up-to-5 in-flight guarantee with its config requirements.

for a principal

Connect idempotence to EOS/transactions, reason about per-session vs cross-session guarantees, and the throughput/ordering trade space.

## What 'in flight' means The producer keeps a queue of `ProduceRequest`s per broker connection. `max.in.flight.requests.per.connection` (default **5**) is how many it will send **without waiting for acks** before it must pause. Higher in-flight means more pipelining and higher throughput; lower means more lock-step. ## How reordering happens Kafka guarantees ordering **only within a partition**, and only as written. Consider two batches headed to the same partition leader, sent back-to-back: - The producer sends **B1**, then **B2** (both in flight, since `max.in.flight > 1`). - **B1** gets a retriable error (e.g. `NotLeaderOrFollowerException`), but **B2** succeeds and is appended. - The producer retries **B1**, which now appends **after** B2. Result: the log holds B2's records before B1's — **reordering within the partition**, even though your application produced them in order. This is purely a producer-side effect of pipelining + retries. ## Old fix: serialize the pipeline Set `max.in.flight.requests.per.connection=1`. Now only one request is outstanding, so a retry of B1 completes before B2 is ever sent. Ordering is preserved — but throughput drops because you lose pipelining and pay a full round-trip per request. ## Modern fix: idempotence Set `enable.idempotence=true` (the **default since Kafka 3.0** when configs are compatible). Mechanism: - The producer is assigned a **Producer ID (PID)** by the broker. - Every record batch per partition carries a **monotonically increasing sequence number**. - The broker tracks the last sequence it accepted per (PID, partition) and **rejects out-of-order or duplicate batches**. Because the broker enforces sequence order, a retried B1 with sequence n cannot be accepted *after* B2 with sequence n+1; the broker would reject B2 as out-of-order (returning `OutOfOrderSequenceException` to the client), and the idempotent producer re-sends in correct sequence. This lets you keep `max.in.flight.requests.per.connection` up to **5** and still guarantee both **no reordering** and **no duplicates** within the producer session. (5 is the hard ceiling the idempotent producer permits; higher values are rejected.) ## Requirements / gotchas - Idempotence requires `acks=all`, `max.in.flight <= 5`, and `retries > 0`. Setting any of these incompatibly while `enable.idempotence=true` throws a `ConfigException`. - Idempotence dedups **within a single producer session**; a producer restart gets a new PID and the guarantee resets (transactions / EOS handle cross-session). - Ordering is still only **per partition** — a custom partitioner or key change spreads records across partitions with no cross-partition order.

  • With enable.idempotence=true, what is the maximum allowed max.in.flight.requests.per.connection and why?
    5. The idempotent producer caps it at 5 because the broker only retains enough sequence-number state per producer/partition to deduplicate and reorder within that window; higher values are rejected with a ConfigException.
  • Does idempotence guarantee ordering across partitions?
    No. Kafka only orders within a single partition. Idempotence preserves per-partition order under retries; records on different partitions have no global ordering.

saying these in an interview costs you the question

  • Saying max.in.flight=1 is the only way to keep ordering — idempotence allows it with up to 5.
  • Claiming Kafka guarantees global ordering across a topic; it's per-partition only.
  • Believing higher in-flight always means reordering even with idempotence on.
  • Thinking idempotence dedups across producer restarts (it's per-session via the PID).

context

resolving links…