skip to content

In KRaft mode, how do you secure the controller quorum, and why is the controller listener security distinct from the broker's client listeners?

level: seniorimportance: must knowfreq 45%

answer

  1. controller.listener.names + listener.security.protocol.map
  2. __cluster_metadata log = ZK's replacement
  3. Carries metadata itself → mTLS/SASL_SSL, not PLAINTEXT
  4. controller.quorum.voters / bootstrap.servers
  5. SCRAM bootstrap via kafka-storage format --add-scram (KIP-900)

basics

~20 s

KRaft controllers form a Raft quorum that replicates the metadata log. You secure their listener (named in controller.listener.names) with its own SSL/SASL entry in listener.security.protocol.map, separate from client listeners, because that channel carries all cluster metadata.

solid answer

~40 s

In KRaft, dedicated controller nodes run a Raft quorum that replicates the __cluster_metadata log — the successor to everything ZooKeeper held. The quorum communicates over a controller listener you name with controller.listener.names (e.g. CONTROLLER), defined in listeners/advertised.listeners, and given a security protocol via listener.security.protocol.map (e.g. CONTROLLER:SSL or CONTROLLER:SASL_SSL). Brokers reach controllers via controller.quorum.voters (or controller.quorum.bootstrap.servers). This listener is distinct from PLAINTEXT/SASL_SSL client listeners because it carries authoritative cluster metadata; if unauthenticated, an attacker reaching it could read or inject metadata records — topic configs, ACLs, SCRAM, feature levels. Hardening: enable TLS (ideally mTLS) and/or SASL on the controller listener, restrict it to a private network, and ensure inter-controller and broker→controller auth principals are authorized. Note KRaft does not yet support delegation tokens and historically had gaps around SCRAM bootstrap.

go deeper

for a junior

Know KRaft replaces ZooKeeper with a controller quorum and that this listener must be secured too.

for a middle

Name the configs: controller.listener.names, listener.security.protocol.map, controller.quorum.voters.

for a senior

Explain why the metadata-carrying listener needs mTLS/SASL_SSL, plus the SCRAM bootstrap (KIP-900) and combined-mode nuances.

for a principal

Architect controller-quorum security with separate principals/keystores, authorizer super.users, network segmentation, and version-aware feature-gap planning.

## What KRaft is **KRaft** (KIP-500) replaces ZooKeeper with a built-in consensus mechanism. A set of **controller** nodes runs the **Raft** protocol to maintain a replicated, ordered log called `__cluster_metadata`. This log *is* the cluster's source of truth: topics, partition assignments, broker registrations, ACLs, SCRAM credentials, feature flags, and producer/transaction state metadata. Brokers consume this log to learn cluster state. The set of controllers eligible to be leader is the **quorum**. ## The controller listener Controllers and brokers don't talk over the normal client listeners. KRaft introduces a **controller listener**: - `controller.listener.names=CONTROLLER` names which listener(s) are the controller endpoint(s). - The endpoint is declared in `listeners` (controllers) and reachable per `controller.quorum.voters=1@host:9093,2@...` (static) or `controller.quorum.bootstrap.servers` (dynamic, KIP-853). - Its wire security is set in `listener.security.protocol.map`, e.g. `CONTROLLER:SASL_SSL`. Traffic on this listener includes Raft vote/fetch requests between controllers and the brokers' metadata fetches from the leader. ## Why it's distinct from client listeners Client listeners (`PLAINTEXT`, `SSL`, `SASL_PLAINTEXT`, `SASL_SSL`) carry produce/consume traffic governed by Kafka ACLs. The **controller listener carries the metadata itself** — the very data those ACLs are stored in. An attacker who can speak to an unsecured controller listener could: - **Read** all metadata, including SCRAM credential records and ACL records. - Potentially **participate in or disrupt** the quorum (impersonate a voter, interfere with Raft) if voter identity isn't authenticated. This is exactly the ZooKeeper trust-root problem relocated into Kafka. So the controller listener deserves *at least* as strong protection as any client listener — typically **mutual TLS** so each controller/broker proves identity, or **SASL_SSL** (e.g. SCRAM or GSSAPI) over TLS. ## Hardening steps 1. **Encrypt + authenticate**: map the controller listener to `SSL` (with `ssl.client.auth=required` for mTLS) or `SASL_SSL`. 2. **Authorize**: with `StandardAuthorizer`, ensure the principals controllers and brokers use are recognized; `super.users` typically lists the controller/broker principals so they aren't denied. 3. **Network isolation**: keep the controller port on a private/management network; never expose it to application clients. 4. **Separate keystores/principals** for the controller listener vs client listeners so a compromised client cert can't reach controllers. ## Edge cases & gaps - **Bootstrapping SCRAM**: in KRaft, SCRAM creds live in the metadata log, creating a chicken-and-egg problem for the very first credentials — addressed by `kafka-storage.sh format --add-scram` (KIP-900) so you can seed credentials at format time. - **No delegation tokens** in early KRaft releases — a feature-parity gap to check per version. - **Combined mode** (a node that is both broker and controller) shares the process but still has a *separate controller listener* you must secure; don't accidentally route client traffic to it. - Misconfiguring `controller.listener.names` to point at a PLAINTEXT listener silently leaves the quorum unauthenticated.

  • How do you provision the very first SCRAM credential in a KRaft cluster when credentials live in the metadata log you can't yet write to?
    Use kafka-storage.sh format with --add-scram (KIP-900) to seed SCRAM credentials into the metadata log at storage-format time, solving the bootstrap chicken-and-egg before the cluster starts.
  • In combined mode where one node is broker+controller, is the controller listener still separate?
    Yes. Even though it's one process, the controller listener is a distinct endpoint named in controller.listener.names and must be secured independently; client traffic should never be routed to it.
  • Why is mTLS often preferred over a single shared SASL secret for the controller quorum?
    mTLS gives each controller/broker a distinct verifiable identity, enabling per-principal authorization and preventing an impersonator from joining the quorum, whereas a shared secret is a single point of compromise.

saying these in an interview costs you the question

  • Treating the controller listener like a client listener and leaving it PLAINTEXT on a 'trusted' network.
  • Assuming KRaft has full security parity with ZooKeeper-mode (delegation tokens were absent early on).
  • Forgetting the SCRAM bootstrap problem and trying to create the first user against a not-yet-running cluster.
  • Believing combined mode removes the need to secure a separate controller listener.

context

resolving links…