skip to content

How is a KafkaPrincipal derived from an authenticated connection, and why would you implement a custom KafkaPrincipalBuilder?

level: seniorimportance: should knowfreq 40%

answer

  1. KafkaPrincipal = type:name (usually User:)
  2. principal.builder.class → KafkaPrincipalBuilder
  3. mTLS default = full cert DN; SASL = username
  4. ssl.principal.mapping.rules to shorten DN
  5. custom builder for groups/OAuth claims/normalization

basics

~20 s

Kafka turns an authenticated connection into a KafkaPrincipal (type:name) via a KafkaPrincipalBuilder. By default mTLS uses the full certificate DN and SASL uses the username. A custom builder lets you remap that to cleaner or group-based principal names.

solid answer

~50 s

Every authenticated connection is mapped to a KafkaPrincipal — a (principalType, name) pair, usually User:<name> — by the class named in principal.builder.class, a KafkaPrincipalBuilder. By default, mTLS (SSL) connections get the full X.500 Distinguished Name from the client certificate (e.g. User:CN=app,OU=eng,O=acme,C=US), while SASL connections get the SASL authentication ID (SCRAM/PLAIN username, or the Kerberos short name). This principal is what both super.users matching and ACLs are evaluated against, so the exact string matters. For mTLS you can avoid brittle full-DN matching using ssl.principal.mapping.rules — regex rules that rewrite the DN into a short name (e.g. extract just the CN). For deeper control — mapping certs to roles/groups, deriving identity from a custom token, or honoring OAuth claims — you implement a custom KafkaPrincipalBuilder. It receives the authentication context and returns the KafkaPrincipal, optionally a KafkaPrincipal carrying group info for group-based ACLs.

go deeper

for a junior

Know that a KafkaPrincipal is the identity (User:name) Kafka checks, derived from the cert or SASL username.

for a middle

Explain the default mTLS-DN vs SASL-username mapping and that ACLs must match exactly.

for a senior

Use ssl.principal.mapping.rules to normalize DNs and justify when a custom KafkaPrincipalBuilder is warranted.

for a principal

Design fleet-wide identity normalization, group/role principals, and OAuth-claim mapping while avoiding identity collisions and ACL-migration breakage.

## From authentication to principal Authentication establishes *who connected*; authorization needs a stable **identity object** to match against `super.users` and ACLs. That object is a **`KafkaPrincipal`** — a simple pair of **type** (almost always `User`) and **name** (a string). It's produced by the class configured in **`principal.builder.class`**, which implements **`KafkaPrincipalBuilder`** (its `build(AuthenticationContext)` returns a `KafkaPrincipal`). ## Default mappings - **mTLS / SSL:** the principal name is the client certificate's full **Distinguished Name (DN)** in RFC2253 form, e.g. `User:CN=app,OU=eng,O=acme,C=US`. This is verbose and order/format-sensitive — every ACL and `super.users` entry must reproduce it exactly. - **SASL (SCRAM/PLAIN):** the principal name is the **username** that authenticated, e.g. `User:app`. - **SASL/GSSAPI (Kerberos):** by default the full principal; `sasl.kerberos.principal.to.local.rules` can shorten it. - **No security / PLAINTEXT:** `User:ANONYMOUS`. ## Taming the DN: ssl.principal.mapping.rules Matching full DNs is brittle, so Kafka provides **`ssl.principal.mapping.rules`** — an ordered list of `RULE:` regex transforms (similar to Kerberos auth_to_local) that rewrite the DN into a friendlier name: ``` ssl.principal.mapping.rules=RULE:^CN=(.*?),.*$/$1/,DEFAULT ``` This extracts the CN, so a cert with `CN=app,OU=eng,...` becomes `User:app`, and your ACLs/super.users can use the short name. `DEFAULT` falls back to the full DN. ## Why a custom KafkaPrincipalBuilder You implement one when the built-in DN/username mapping plus regex rules aren't enough: - **Role/group-based authorization:** return a principal annotated with groups (Kafka supports group principals so ACLs can be granted to a group), derived from the cert's OU or an external directory lookup. - **OAuth/OIDC (SASL/OAUTHBEARER):** map a verified JWT's `sub` or a custom claim to the principal name, rather than the raw token subject. - **Custom/legacy auth schemes:** translate an internal token or header into a canonical identity. - **Normalization:** enforce a single canonical identity format across mTLS and SASL clients so ACLs are written once. The builder gets the `AuthenticationContext` (SSL session with peer certs, or SASL server with the authorization ID) and returns the `KafkaPrincipal`. It runs **after** authentication succeeds — it cannot grant access by itself; it only shapes the identity that the authorizer then evaluates. ## Edge cases / pitfalls - A super user that won't take effect almost always means the produced principal string differs from the `super.users` entry — check DN ordering, mapping rules, and type. - Mapping rules apply to the *entire fleet's* certs; an overly broad rule can collapse two distinct DNs to the same name (identity collision). - The builder runs on the hot path of every new connection; keep external lookups cached. - Changing the principal mapping invalidates existing ACLs that referenced the old names — plan migrations.

  • Your mTLS clients authenticate but every ACL fails to match. What's the most likely cause and fix?
    ACLs/super.users were written with a short name (e.g. User:app) but the default principal is the full DN (User:CN=app,OU=...). Fix by adding ssl.principal.mapping.rules to extract the CN, or rewrite ACLs to the exact DN. The strings must match exactly.
  • How do you do group-based authorization in Kafka?
    Use a custom KafkaPrincipalBuilder that returns a KafkaPrincipal carrying group membership (KafkaPrincipal supports a primary user plus group info / KafkaPrincipalSerde), derived from the cert OU or a directory lookup, then grant ACLs to the group principal.

saying these in an interview costs you the question

  • Saying the principal builder performs authorization — it only constructs identity; the authorizer decides access.
  • Assuming mTLS principal is the CN by default — it is the full DN unless mapping rules are applied.
  • Forgetting that principal strings must match super.users/ACLs exactly (DN order/format sensitivity).
  • Claiming you can't map OAuth claims or groups — a custom KafkaPrincipalBuilder handles both.

context

resolving links…