What does unclean.leader.election.enable do, and how does it change the durability/availability trade-off when combined with the other replication settings?
answer
- Out-of-sync replica becomes leader?
- Default false = consistency, offline partition
- True = available but truncates/loses committed data
- Final availability-vs-durability lever
- Flipped to false in 0.11+
basics
~20 sIt decides whether a partition can elect a leader from a replica that was NOT in the in-sync set when all in-sync replicas are down. Enabled = stay available but possibly lose data; disabled (default) = stay consistent but the partition goes offline until an in-sync replica returns.
solid answer
~40 sunclean.leader.election.enable controls leader election when no in-sync replica (ISR member) is available. With it disabled (the modern default, false), if every ISR replica is down, the partition becomes leaderless and offline for both reads and writes until an ISR member comes back — preserving consistency and never serving data that wasn't fully replicated. With it enabled (true), Kafka may promote an out-of-sync replica to leader; the partition stays available, but that replica is behind the old leader, so any records acknowledged only on the now-dead ISR members are permanently lost and the log effectively truncates. It is the final availability-vs-durability lever: even RF=3/MISR=2/acks=all can lose data if unclean election is enabled and you hit the worst case. Keep it false for durable topics; only consider true for low-value, availability-first data.
go deeper
Know it's the on/off switch for letting a behind replica become leader, and that false is the safe default.
Explain the offline-vs-data-loss outcome and why default is false.
Tie it to acks/MISR as the final lever and reason about the catastrophic scenario and operational override.
Set org policy per topic tier, integrate with multi-DC/rack awareness and runbooks for emergency recovery.
## Leader election background Each partition has one leader; followers replicate from it. The **ISR** is the set of replicas fully caught up to the leader. When a leader fails, the controller picks a new leader. - A ***clean* election** chooses a replica that was in the ISR — guaranteed to have every committed (acknowledged) record. - An ***unclean* election** chooses a replica that was NOT in the ISR — one that may be missing recent committed records. ## The setting `unclean.leader.election.enable` (broker default and per-topic override) decides whether unclean elections are permitted. The default flipped to **false** in Kafka 0.11+ to favor correctness. ## Disabled (false) — consistency over availability Suppose RF=3 and over time all replicas except a lagging one fail, so the ISR is empty (or only-failed members remain). With unclean disabled, the controller refuses to elect the out-of-sync replica. The partition has no leader: producers and consumers both get errors (the partition is offline). It stays offline until one of the previously in-sync replicas restarts and can become leader. You lose availability but never serve or acknowledge a divergent log — no committed record vanishes. ## Enabled (true) — availability over consistency In the same scenario, the controller promotes the out-of-sync replica to leader. The partition is immediately writable/readable again. But that replica's log ends earlier than the failed leader's: every record that was committed (acknowledged under acks=all on the old ISR) but not yet replicated to this replica is **gone**. Worse, consumers that had read those records now see a log that no longer contains them — a form of data loss and divergence. When the old leader returns it must truncate its log to match the new leader, discarding those records. ## Interaction with the other knobs acks=all + MISR=2 guarantee a committed record is on >= 2 replicas, which makes *clean* recovery possible as long as one of those replicas survives. Unclean election undermines that guarantee only in the catastrophic case where *all* replicas that held the record are simultaneously down and a stale one is promoted. So the durability of the whole system is: - (acks + MISR define what 'committed' means) AND - (unclean=false ensures only committed-complete replicas can lead). Turning unclean on re-introduces loss the other settings were protecting against. ## Edge cases & operations - (1) Even with unclean=false, a total outage of all ISR members makes the partition unavailable — operators sometimes temporarily flip unclean=true to force recovery, accepting loss, then flip it back. - (2) The KRaft controller (post-ZooKeeper) honors the same setting. - (3) Monitor `OfflinePartitionsCount`; a nonzero value with unclean=false signals partitions waiting for an in-sync replica. - (4) For multi-DC stretch clusters, leaving unclean=false plus proper rack/DC awareness prevents a network partition from silently truncating data on the losing side.
- Can a topic with RF=3, min.insync.replicas=2, acks=all still lose acknowledged data? When?Yes, only if unclean.leader.election.enable=true and you reach a state where all replicas holding a committed record are down and a stale out-of-sync replica is elected leader. With unclean=false the partition goes offline instead, preserving the data.
- When might an operator deliberately enable unclean leader election temporarily?During a severe outage where all in-sync replicas for a partition are down and won't recover soon, to bring the partition back online at the cost of losing the most recent unreplicated records — then disable it again afterward.
saying these in an interview costs you the question
- Saying unclean election just speeds up failover with no downside (it can permanently lose committed data).
- Believing the default is true (it has been false since 0.11).
- Claiming acks=all alone prevents all loss regardless of unclean.leader.election.enable.