What is the difference between the static controller.quorum.voters config and the dynamic quorum introduced by KIP-853?
answer
- static = controller.quorum.voters on every node
- dynamic = voter set in the metadata log
- controller.quorum.bootstrap.servers is just a hint
- AddRaftVoter / RemoveRaftVoter at runtime
- GA in Kafka 3.9
basics
~10 sStatic quorums list every controller in controller.quorum.voters on all nodes, fixed at startup. KIP-853 dynamic quorums (KRaft) let you add or remove controllers at runtime with AddRaftVoter/RemoveRaftVoter instead, using controller.quorum.bootstrap.servers.
solid answer
~40 sBefore KIP-853, KRaft controller membership was static: every broker and controller hard-coded the full voter set in controller.quorum.voters (id@host:port for each voter). Changing membership meant editing config on every node and rolling the cluster, which made replacing a failed controller painful and risky. KIP-853 (GA in Kafka 3.9 for new clusters, formats with kafka-storage --standalone or --no-initial-controllers) makes the voter set dynamic state stored in the cluster metadata log itself. Nodes discover the quorum via controller.quorum.bootstrap.servers (a connection hint, not the authoritative voter list). Operators then change membership at runtime with kafka-metadata-quorum add-controller / remove-controller, which issue AddRaftVoter/RemoveRaftVoter RPCs. The static and dynamic models are mutually exclusive per cluster, decided at format time.
go deeper
Know that static lists all controllers in one config; dynamic lets you add/remove controllers while running.
Explain bootstrap.servers as a hint vs. authoritative log-stored voter set, and the AddRaftVoter/RemoveRaftVoter RPCs.
Discuss format-time mode selection (--standalone / --no-initial-controllers), directory.id, and operational pain the static model caused.
Reason about migration constraints, why membership-as-log-state is safer than config drift, and version/GA boundaries (3.9).
## Background Kafka clusters need a metadata layer that tracks topics, partitions, configs, ACLs, and broker registrations. Historically this was ZooKeeper; **KRaft** (KIP-500) replaced ZooKeeper with an internal Raft-based consensus protocol. A small set of **controller** nodes form a **Raft quorum**: they elect a leader, and metadata changes are committed to a replicated log (`__cluster_metadata`) once a majority of voters acknowledge them. A majority of N voters is `floor(N/2)+1`, so 3 voters tolerate 1 failure, 5 tolerate 2. ## The static model (pre-KIP-853) In early KRaft (Kafka 3.0–3.8), the **voter set was static configuration**. Every node — brokers and controllers alike — declared the complete quorum in `controller.quorum.voters`, formatted as `1@host1:9093,2@host2:9093,3@host3:9093`. The IDs and endpoints were fixed at startup. Consequences: - To replace a dead controller you had to reuse its exact node id and endpoint, or edit config everywhere and roll the whole cluster. - You could not safely grow a 3-controller quorum to 5, or shrink it, while online. - The config was duplicated and had to stay consistent across all nodes — drift caused split-brain risk. ## The dynamic model (KIP-853) KIP-853, **Dynamic KRaft Quorum Reconfiguration**, makes membership **runtime state inside the metadata log** rather than static config. Key pieces: - `controller.quorum.bootstrap.servers` — a list of host:port endpoints used only to *find* the quorum and discover the current leader. It is a hint, not the source of truth, so it can be incomplete or stale. - The authoritative voter set lives in the log as `VotersRecord`, replicated like any other metadata. - Each voter has a **`directory.id`** (a UUID for its metadata log directory) in addition to its node id, so a voter is uniquely identified even if a node id is reused. - Membership changes happen at runtime via `AddRaftVoter` and `RemoveRaftVoter` RPCs, exposed through `kafka-metadata-quorum --command add-controller / remove-controller`. ## Choosing the mode The two modes are mutually exclusive and chosen at **format time** with `kafka-storage format`: - Static: provide `controller.quorum.voters`. - Dynamic: format the first controller with `--standalone` (or a multi-node initial set with `--initial-controllers`), and use `controller.quorum.bootstrap.servers`. New controllers are formatted with `--no-initial-controllers` and then joined via add-controller. Dynamic quorums became production-ready in **Kafka 3.9** for new clusters. Existing static clusters cannot be flipped in place to dynamic in 3.9 — that path matured later. ## Edge cases - Mixing `controller.quorum.voters` and `controller.quorum.bootstrap.servers` in dynamic mode is an error; bootstrap.servers replaces the static list. - bootstrap.servers being stale is fine as long as at least one listed endpoint can reach a live voter that knows the current leader.
- Which config replaces controller.quorum.voters in the dynamic model?controller.quorum.bootstrap.servers — a discovery hint listing some controller endpoints; the authoritative voter set lives in the metadata log, not in this config.
- In which Kafka release did dynamic quorums become production-ready for new clusters?Kafka 3.9. Static clusters could not be converted in place at that point.
saying these in an interview costs you the question
- Saying controller.quorum.bootstrap.servers is the authoritative voter list — it is only a discovery hint.
- Claiming you can flip an existing static cluster to dynamic just by changing config in 3.9.
- Thinking dynamic quorums removed the majority-quorum requirement — Raft still needs a majority to commit.