skip to content

During failover, what is unclean leader election, what does it trade off, and when (if ever) would you enable it?

level: seniorimportance: should knowfreq 48%

answer

  1. out-of-sync replica promoted
  2. default false
  3. availability vs durability
  4. acked data can be lost; logs truncate down
  5. enable per-topic for reproducible data only

basics

~20 s

Unclean leader election lets an out-of-sync replica become leader when no in-sync replica survives. It restores availability but can lose acknowledged data. It's off by default; enable it only when uptime matters more than durability.

solid answer

~50 s

Normally Kafka elects a new leader only from the ISR, which guarantees acknowledged data survives. But if every ISR member is also down, the partition goes offline and stays offline until an ISR replica returns. Unclean leader election (unclean.leader.election.enable, default false) relaxes this: the controller may promote a surviving but out-of-sync replica. That replica is missing records the old leader had acknowledged, so those records — and any consumer offsets past them — are silently lost; followers and consumers truncate to the new leader's (shorter) log. The trade-off is the classic CAP-flavored choice: availability vs durability. You enable it for data where staying up beats never losing a record — e.g. high-volume metrics or logs that are reproducible — and keep it off for anything where acknowledged-means-durable is a hard requirement (payments, audit, event-sourced state).

go deeper

for a junior

Know it lets a behind replica become leader and can lose data; it's off by default.

for a middle

Explain the offline-partition deadlock it resolves and the availability/durability trade-off.

for a senior

Discuss truncation of returning brokers, consumer offset rewind, and per-topic scoping decisions.

for a principal

Frame as a durability policy across the platform; combine rack-awareness, min.insync.replicas, and per-topic settings to avoid ever needing it on critical topics.

## The default safe behavior Kafka, by default, elects a new leader **only from the ISR** (in-sync replica set). Because `acks=all` + `min.insync.replicas` ensures every acknowledged record is on every ISR member, electing from the ISR guarantees **no acknowledged data loss**. ## The deadlock case What if **all ISR members are down** at once (e.g. a rack/AZ failure took out every in-sync copy)? With the safe policy, the partition has **no eligible leader**, so it goes **offline**: producers and consumers for that partition get errors and the partition is unavailable until at least one former ISR member comes back online and can be elected. ## What unclean leader election does `unclean.leader.election.enable=true` (default `false`; settable per-topic or cluster-wide) tells the controller it *may* elect a leader from replicas that are **not in the ISR** — i.e. a replica that is alive but **behind**. This restores availability immediately. ### The cost The promoted replica's log is **shorter** than the old leader's was. Records the old leader had appended (and possibly acknowledged to producers) are **not present** on the new leader. When the dead brokers return, their followers **truncate** their longer logs down to match the new (shorter) leader — permanently discarding those records. Consumers that had read past the new leader's end will see offsets effectively rewound; this is **silent data loss** plus possible duplicate/out-of-order reprocessing. ## The trade-off framed This is an availability-vs-durability decision: - **Off (default):** prefer **durability/consistency** — partition may be unavailable, but acknowledged data is never lost. - **On:** prefer **availability** — partition recovers fast, but you may lose acknowledged records. ## When to enable Enable only when **uptime clearly outranks per-record durability** and the data is **reproducible or tolerant of loss**: clickstream, metrics, application logs, cache-warming streams. Often scoped **per topic** so only those topics relax the guarantee. Keep it **off** for: payments, ledgers, audit trails, event-sourced aggregates, anything where 'acknowledged means durable' is contractual. ## Related levers - `min.insync.replicas` + `acks=all`: the durability contract unclean election would violate. - Spreading replicas across racks/AZs (`broker.rack`) reduces the chance the *entire* ISR dies together, lowering the temptation to enable unclean election. ## Common mistake Leaving it globally `true` 'so we never have downtime' — that silently weakens durability for every topic, including the ones that can't tolerate loss.

  • What happens to a partition if unclean leader election is disabled and every ISR member is down?
    The partition goes offline — no eligible leader exists — and stays unavailable to producers and consumers until at least one former ISR replica comes back online and can be elected. No data is lost, but availability is sacrificed.
  • After an unclean election, what happens to the returning brokers that had a longer log?
    They become followers of the new (out-of-sync) leader and truncate their logs down to the new leader's end, permanently discarding the extra records the old leader had — that's the data loss. Consumers may see offsets effectively rewound.

saying these in an interview costs you the question

  • Saying unclean election is on by default (it's false by default in modern Kafka).
  • Claiming it never loses data — losing acknowledged data is exactly its trade-off.
  • Recommending it cluster-wide as a generic 'high availability' setting.
  • Confusing it with min.insync.replicas — that's the write-side durability contract, not the election policy.

context

resolving links…