skip to content

How does listener.security.protocol.map work, and how do you use named listeners to run multiple security protocols on one broker?

level: middleimportance: should knowfreq 50%

answer

  1. name != protocol
  2. custom names require the map
  3. every referenced name must be mapped
  4. listener.name.<name>.* per-listener config
  5. internal SSL + external SASL_SSL on one broker

basics

~20 s

listener.security.protocol.map maps each named listener to one of the four security protocols. With custom listener names you must define this map; it lets one broker expose, for example, an SSL internal listener and a SASL_SSL external listener at the same time.

solid answer

~40 s

A listener has a **name** and an address. The four built-in names (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL) implicitly map to their matching protocol. But if you invent custom names like INTERNAL or EXTERNAL — which you must, to run two listeners of the *same* protocol or to give them meaningful roles — Kafka doesn't know what security each implies, so you provide `listener.security.protocol.map=INTERNAL:SSL,EXTERNAL:SASL_SSL,BROKER:PLAINTEXT`. Every name appearing in `listeners`/`advertised.listeners`/`inter.broker.listener.name`/`control.plane.listener.name` must have an entry. This lets a single broker simultaneously serve different protocols per network segment: e.g. plaintext for a localhost-only admin path, SSL for in-VPC traffic, SASL_SSL for the internet-facing edge. Each protocol then has its own SSL/SASL configs, often scoped per listener using the `listener.name.<name>.*` prefix.

go deeper

for a junior

Know the map associates listener names with one of the four protocols.

for a middle

Configure a two-listener broker (internal/external) with different protocols and per-listener prefixed configs.

for a senior

Design segment-specific hardening and reason about which listener inter-broker and controller traffic should use.

for a principal

Define org-wide listener naming/hardening conventions and govern dynamic per-listener config updates safely.

**Listener names vs protocols.** In Kafka, a *listener* is an endpoint with a **name** and a **bind address**. The security protocol is a separate attribute. For the four reserved names (PLAINTEXT, SSL, SASL_PLAINTEXT, SASL_SSL), the name *is* the protocol, so no extra mapping is needed. But as soon as you want **two listeners using the same protocol**, or want **role-descriptive names**, you must use custom names — and then you must tell Kafka which protocol each name uses. **The map.** `listener.security.protocol.map` is a comma-separated list of `LISTENER_NAME:PROTOCOL` pairs: ``` listener.security.protocol.map=INTERNAL:SSL,EXTERNAL:SASL_SSL,CONTROLLER:PLAINTEXT ``` Rules: - Every name referenced anywhere — in `listeners`, `advertised.listeners`, `inter.broker.listener.name`, and (KRaft) `controller.listener.names` / `control.plane.listener.name` — **must** appear in the map, or the broker fails to start. - The value must be one of the four protocols. - Names are case-insensitive but conventionally uppercase. **Why this enables multi-protocol brokers.** A single broker can bind several listeners, each with its own protocol, serving distinct network segments with distinct hardening: ``` listeners=INTERNAL://0.0.0.0:9092,EXTERNAL://0.0.0.0:9094 advertised.listeners=INTERNAL://kafka.svc:9092,EXTERNAL://kafka.example.com:9094 listener.security.protocol.map=INTERNAL:SSL,EXTERNAL:SASL_SSL inter.broker.listener.name=INTERNAL ``` Here internal traffic and broker-to-broker replication use mTLS (SSL), while external clients must authenticate via SASL inside TLS (SASL_SSL). This is the **per-network hardening** pattern: weaker-but-faster security inside a trusted boundary, stronger auth at the edge. **Per-listener configuration.** Because each listener may need different keystores, truststores, enabled SASL mechanisms, or client-auth settings, Kafka supports **listener-prefixed configs**: `listener.name.<lowercased-name>.<property>`. Examples: ``` listener.name.external.sasl.enabled.mechanisms=SCRAM-SHA-512 listener.name.external.ssl.keystore.location=/certs/external.keystore.jks listener.name.internal.ssl.client.auth=required ``` This lets the EXTERNAL listener require SCRAM auth while the INTERNAL listener uses mTLS only, all on the same broker. Some of these are dynamically updatable via the AdminClient/kafka-configs without a restart. **Edge cases:** - If you reuse a reserved name (e.g. SSL) as a custom listener name but want a *different* protocol, the map entry can override it — but this is confusing; prefer unique custom names. - `inter.broker.listener.name` must reference a listener whose protocol the brokers can mutually authenticate with; you cannot point it at a listener clients also use if that breaks the broker's own credentials. - In KRaft mode the controller listener is separate and also needs a map entry.

  • You want two listeners both using SASL_SSL but with different keystores. How?
    Give them distinct custom names (e.g. INTERNAL, EXTERNAL), map both to SASL_SSL in listener.security.protocol.map, then use listener.name.internal.* and listener.name.external.* prefixed configs for the separate keystores/mechanisms.
  • What happens if a name in listeners isn't present in listener.security.protocol.map?
    The broker fails to start with a configuration error, because Kafka cannot determine the security protocol for that listener (unless the name is one of the four reserved ones).

saying these in an interview costs you the question

  • Believing a listener's name automatically determines its protocol even for custom names
  • Forgetting that inter.broker and controller listener names also need map entries
  • Thinking one broker can only run a single security protocol
  • Using global ssl.* configs when you need per-listener keystores (should use listener.name.<name>.* prefix)

context

resolving links…