How does a producer decide which partition a record goes to, and how does that interact with ordering guarantees?

seniormust knowfreq 72%

answer

Explicit partition > key-hash > sticky/round-robin
murmur2(key) % numPartitions
Same key → same partition → order preserved
Adding partitions breaks key→partition mapping
Idempotence + max.in.flight<=5 keep order on retry

basics

~20 s

If a record has a key, the default partitioner hashes the key (murmur2 mod partition count) so the same key always maps to the same partition, preserving per-key order. Keyless records are spread across partitions (sticky/round-robin), so their relative order across partitions isn't guaranteed.

solid answer

~50 s

Partition selection happens in the producer. Precedence: an explicit partition on the ProducerRecord wins; otherwise if a key is present, the default partitioner computes murmur2(key) % numPartitions, so identical keys consistently land in the same partition; otherwise (no key) the modern default uses a sticky strategy that batches keyless records to one partition per batch, rotating between batches for even spread (older clients used round-robin). Ordering is guaranteed only within a partition, so keying is the lever for ordering: route all records that must stay ordered (e.g. one customer's events) to the same key → same partition. The catch: this hash maps over the current partition count, so increasing partitions later changes the key→partition mapping and breaks ordering for in-flight keys. Also, with retries you need max.in.flight.requests.per.connection <= 5 plus idempotence (enable.idempotence=true) to keep per-partition order on retry; otherwise a retried batch can be reordered.

Partition selection happens in the producer. Precedence: an explicit `partition` on the `ProducerRecord` wins; otherwise if a key is present, the default partitioner computes `murmur2(key) % numPartitions`, so identical keys consistently land in the same partition; otherwise (no key) the modern default uses a **sticky** strategy that batches keyless records to one partition per batch, rotating between batches for even spread (older clients used round-robin). Ordering is guaranteed only **within a partition**, so keying is the lever for ordering: route all records that must stay ordered (e.g. one customer's events) to the same key → same partition. The catch: this hash maps over the *current* partition count, so increasing partitions later changes the key→partition mapping and breaks ordering for in-flight keys. Also, with retries you need `max.in.flight.requests.per.connection <= 5` plus idempotence (`enable.idempotence=true`) to keep per-partition order on retry; otherwise a retried batch can be reordered.

go deeper

for a junior

Know that same key goes to same partition and that keeps those records in order.

for a middle

Explain the explicit > key-hash > sticky precedence and that ordering holds only within a partition.

for a senior

Reason about murmur2 % N, the repartitioning hazard, and retry-induced reordering with idempotence/in-flight settings.

for a principal

Design keying strategy and partition stability policy for order-sensitive domains and teach the producer ordering invariants.

The problem

Kafka only orders records within a single partition. So the question "which partition does a record go to?" is the same as "what ordering will my consumers see?" The producer answers it.

Partition selection precedence (producer side)

For each ProducerRecord(topic, [partition], [key], value):

Explicit partition — if the record specifies a partition number, it is used verbatim.
Key present — the default partitioner (DefaultPartitioner) computes a hash of the serialized key with Kafka's murmur2 and takes it modulo the number of partitions: partition = murmur2(keyBytes) % numPartitions. Consequence: the same key always maps to the same partition (given a fixed partition count), so all records for that key are in one ordered log.
No key — modern clients (KIP-480, since 2.4) use the sticky partitioner: fill a whole batch for one partition, then move to another partition for the next batch. This keeps batches large (better throughput) while spreading load over time. Older clients used plain round-robin per record.

You can also plug a custom Partitioner via partitioner.class.

Ordering consequences

Within a key: because a key is sticky to one partition, records sharing a key are strictly ordered. This is how you get per-entity ordering (e.g. all events for userId=42 in order) without sacrificing cluster-wide parallelism.
Across keys / keyless: records for different keys may land in different partitions and have no defined relative order. Keyless records spread across partitions are explicitly unordered relative to each other.

The repartitioning trap

The mapping is murmur2(key) % N. If you grow N (add partitions), the modulo changes and a given key may map to a different partition. Records for that key now exist in two partitions — old ones in the original, new ones elsewhere — and their global order is lost. (Sizing/repartitioning decisions are owned by a sibling topic; the fundamental point here is that key→partition stability depends on a fixed partition count.)

Retries and in-flight ordering

Even within one partition, ordering can break on retry if multiple requests are in flight: batch 2 could be acked before a retried batch 1, reordering the log. Two settings protect order:

enable.idempotence=true (default true in recent clients) — the broker dedups and preserves order even with retries.
max.in.flight.requests.per.connection <= 5 — required for idempotence to guarantee ordering; with idempotence off, you'd need it set to 1 to be safe.

Practical recipe

Need per-entity ordering? Use that entity's id as the key.
Need maximum spread and don't care about order? Leave the key null.
Never assume order across partitions, and avoid changing partition count for keyed, order-sensitive topics.

Code

ProducerRecord<String,String> r = new ProducerRecord<>("orders", "user-42", payload);
// key "user-42" -> murmur2("user-42") % numPartitions -> always same partition

## The problem Kafka only orders records **within a single partition**. So the question "which partition does a record go to?" is the same as "what ordering will my consumers see?" The producer answers it. ## Partition selection precedence (producer side) For each `ProducerRecord(topic, [partition], [key], value)`: 1. **Explicit partition** — if the record specifies a partition number, it is used verbatim. 2. **Key present** — the default partitioner (`DefaultPartitioner`) computes a hash of the serialized key with Kafka's **murmur2** and takes it modulo the number of partitions: `partition = murmur2(keyBytes) % numPartitions`. Consequence: the **same key always maps to the same partition** (given a fixed partition count), so all records for that key are in one ordered log. 3. **No key** — modern clients (KIP-480, since 2.4) use the **sticky partitioner**: fill a whole batch for one partition, then move to another partition for the next batch. This keeps batches large (better throughput) while spreading load over time. Older clients used plain round-robin per record. You can also plug a custom `Partitioner` via `partitioner.class`. ## Ordering consequences - **Within a key:** because a key is sticky to one partition, records sharing a key are strictly ordered. This is how you get per-entity ordering (e.g. all events for `userId=42` in order) without sacrificing cluster-wide parallelism. - **Across keys / keyless:** records for different keys may land in different partitions and have **no defined relative order**. Keyless records spread across partitions are explicitly unordered relative to each other. ## The repartitioning trap The mapping is `murmur2(key) % N`. If you grow `N` (add partitions), the modulo changes and a given key may map to a **different** partition. Records for that key now exist in two partitions — old ones in the original, new ones elsewhere — and their global order is lost. (Sizing/repartitioning decisions are owned by a sibling topic; the fundamental point here is that key→partition stability depends on a fixed partition count.) ## Retries and in-flight ordering Even within one partition, ordering can break on **retry** if multiple requests are in flight: batch 2 could be acked before a retried batch 1, reordering the log. Two settings protect order: - `enable.idempotence=true` (default true in recent clients) — the broker dedups and **preserves order** even with retries. - `max.in.flight.requests.per.connection <= 5` — required for idempotence to guarantee ordering; with idempotence off, you'd need it set to 1 to be safe. ## Practical recipe - Need per-entity ordering? Use that entity's id as the key. - Need maximum spread and don't care about order? Leave the key null. - Never assume order across partitions, and avoid changing partition count for keyed, order-sensitive topics. ### Code ``` ProducerRecord<String,String> r = new ProducerRecord<>("orders", "user-42", payload); // key "user-42" -> murmur2("user-42") % numPartitions -> always same partition ```

Why can increasing partition count break ordering for an existing key?
Because the default partitioner uses murmur2(key) % N. Changing N changes the modulo, so the key may now hash to a different partition. New records go to the new partition while old ones remain in the original, so the two are no longer in a single ordered log.
How do retries threaten per-partition ordering, and how do you prevent it?
With multiple in-flight requests, a later batch can be acknowledged before a retried earlier batch, reordering the log. Setting enable.idempotence=true (with max.in.flight.requests.per.connection <= 5) makes the broker preserve order and dedup across retries.

saying these in an interview costs you the question

Saying keyless records preserve order across the topic
Believing partition assignment is random even when a key is set
Ignoring that adding partitions rehashes key→partition
Assuming retries never reorder records within a partition
Claiming Kafka uses default Java hashCode for keys (it uses murmur2)

How does a producer decide which partition a record goes to, and how does that interact with ordering guarantees?

§ answer

basics

solid answer

§ go deeper

▾level notes4 levels

▾deep dive~2 min

The problem

Partition selection precedence (producer side)

Ordering consequences

The repartitioning trap

Retries and in-flight ordering

Practical recipe

Code

▾common follow-ups2 questions

▾red flags5 flags

§ context

▾related questions5 questions

▾related topics3 topics

The problem

Partition selection precedence (producer side)

Ordering consequences

The repartitioning trap

Retries and in-flight ordering

Practical recipe

Code

answer

go deeper

context