In KRaft mode, what is the active controller and what role does it play in managing cluster metadata?

## Metadata, and where it lives Apache Kafka stores not just user data but also *metadata* — which topics exist, how many partitions each has, where partition replicas live, which brokers are alive, ACLs, configs, and so on. Historically this metadata lived in ZooKeeper, an external coordination service. **KRaft** (KRaft = Kafka Raft Metadata mode, introduced by KIP-500) removes ZooKeeper and stores metadata inside Kafka itself. ## The controllers In KRaft you designate certain nodes with `process.roles=controller` (or `broker,controller` for combined nodes). These controller nodes form a *quorum* — typically 3 or 5 — that runs the Raft consensus protocol over an internal topic named `__cluster_metadata` (a single-partition replicated log). ## The active controller Raft elects exactly one leader of that log; in Kafka terminology this leader is the **active controller**. It is the *only* node permitted to *write* metadata. Every metadata change is expressed as a record — for example: - a `TopicRecord` (a new topic), - a `PartitionRecord` (a partition's replica assignment and leader), - or a `BrokerRegistration`/`RegisterBrokerRecord` (a broker joining). The active controller appends these records, in order, to the metadata log. ## Followers and brokers - The other controllers are Raft *followers*; they replicate the log so they are ready to take over. - Brokers (the nodes that actually serve produce/fetch traffic) act as *observers*: they continuously fetch the metadata log and **replay** it to rebuild their in-memory picture of the cluster. This is an ***event-sourced state machine***: state = result of applying the ordered stream of records. ## Why a single writer Funneling all writes through one node and one ordered log means there is one authoritative order of events. Every node that replays the same prefix of the log reaches the same state, so the cluster converges to a consistent view without distributed locks or a separate coordination system. ## Failover If the active controller crashes, the Raft quorum holds an election and promotes an up-to-date follower to active controller. Because followers already have the replicated log, the new leader can resume appending almost immediately. A **majority (quorum)** of controllers must be available for writes to proceed. ## Edge cases to know - a record is only durable/committed once a majority of the quorum has it; - brokers that lag are eventually consistent (they catch up by replaying); - and the active controller is a logical role distinct from a partition *leader* for user data — they are different leadership concepts.