skip to content

How does a Kafka broker validate an OAUTHBEARER JWT, and which sasl.oauthbearer.* configs control it?

level: seniorimportance: must knowfreq 45%

answer

  1. JWKS endpoint -> public keys -> verify signature
  2. expected.issuer, expected.audience, exp + clock skew
  3. sub.claim.name => principal
  4. OAuthBearerValidatorCallbackHandler on broker
  5. KIP-768 native OIDC validator

basics

~10 s

The broker fetches the IdP's public keys from a JWKS endpoint and checks the JWT's signature, issuer, audience, and expiry. Configs like sasl.oauthbearer.jwks.endpoint.url, expected.issuer, expected.audience, and sub.claim.name control this.

solid answer

~30 s

On the broker, the server callback handler `OAuthBearerValidatorCallbackHandler` validates incoming JWTs. It retrieves the IdP signing keys from `sasl.oauthbearer.jwks.endpoint.url` (a JWKS document of public keys) and verifies the token signature, caching keys and periodically refreshing them (`sasl.oauthbearer.jwks.endpoint.refresh.ms`). It enforces `sasl.oauthbearer.expected.issuer` against the `iss` claim and `sasl.oauthbearer.expected.audience` against `aud`, and rejects expired tokens using `exp` with optional `sasl.oauthbearer.clock.skew.seconds` tolerance. The authenticated principal name comes from `sasl.oauthbearer.sub.claim.name` (default `sub`); `scope.claim.name` can carry scopes. For air-gapped setups, `sasl.oauthbearer.jwks.endpoint.url` can point at a file:// JWKS. This validation runs per new connection during the SASL handshake; once authenticated, the principal feeds the authorizer for ACL checks.

go deeper

for a junior

Know the broker checks the token's signature and expiry before trusting it.

for a middle

Name JWKS-based signature verification plus issuer/audience/expiry checks and the principal claim.

for a senior

Walk through every validation step and the relevant sasl.oauthbearer.* configs, including clock skew and key refresh.

for a principal

Reason about JWKS availability/key-rotation failure modes, opaque-token limitations, and air-gapped (file:// JWKS) deployments at fleet scale.

## Where validation happens With OAUTHBEARER there are two callback handlers. The **client** uses a login callback to *acquire* a token; the **broker** uses a server/validation callback to *verify* it. Apache Kafka's broker-side validator is `org.apache.kafka.common.security.oauthbearer.OAuthBearerValidatorCallbackHandler`, configured per listener via `listener.name.<listener>.oauthbearer.sasl.server.callback.handler.class`. ## What a JWT contains A JWT has a header (alg + key id `kid`), a payload of **claims**, and a signature. Relevant claims: - `iss` — issuer (which IdP minted it) - `aud` — audience (who the token is for) - `exp` — expiry (epoch seconds) - `iat`/`nbf` — issued-at / not-before - `sub` — subject (the identity) - `scope` — granted scopes ## The validation steps 1. **Signature**: The validator needs the IdP's public key. It fetches a **JWKS** (JSON Web Key Set) from `sasl.oauthbearer.jwks.endpoint.url`. The JWKS lists public keys each with a `kid`; the validator matches the token header's `kid` to a key and verifies the RSA/EC signature. Keys are cached and refreshed every `sasl.oauthbearer.jwks.endpoint.refresh.ms` (default 1h), with retry backoff configs. 2. **Issuer**: `iss` must equal `sasl.oauthbearer.expected.issuer`. 3. **Audience**: `aud` must contain `sasl.oauthbearer.expected.audience` (a list). 4. **Expiry / timing**: `exp` must be in the future; `sasl.oauthbearer.clock.skew.seconds` allows small clock differences between broker and IdP. 5. **Principal extraction**: the principal name is read from the claim named by `sasl.oauthbearer.sub.claim.name` (default `sub`). `sasl.oauthbearer.scope.claim.name` (default `scope`) maps scopes. ## Important configs | Config | Purpose | |---|---| | `sasl.oauthbearer.jwks.endpoint.url` | Where to fetch signing public keys (http(s) or file://) | | `sasl.oauthbearer.jwks.endpoint.refresh.ms` | Key cache refresh interval | | `sasl.oauthbearer.expected.issuer` | Required `iss` value | | `sasl.oauthbearer.expected.audience` | Required `aud` value(s) | | `sasl.oauthbearer.sub.claim.name` | Claim used as principal | | `sasl.oauthbearer.scope.claim.name` | Claim used for scopes | | `sasl.oauthbearer.clock.skew.seconds` | Allowed clock drift on `exp`/`nbf` | ## Edge cases and pitfalls - **JWKS unreachable**: if the broker can't fetch keys (network/firewall), all new OAUTHBEARER auths fail. Cache + refresh mitigate transient outages; pre-warming or file:// JWKS helps in locked-down environments. - **Key rotation**: when the IdP rotates signing keys, the broker must refresh JWKS before tokens signed with the new key arrive; otherwise it sees an unknown `kid` and rejects. - **Opaque vs JWT tokens**: the native validator expects a self-contained JWT. Opaque tokens that require IdP introspection (RFC 7662) are not handled by the built-in validator — you'd need a custom server callback or a library like Strimzi kafka-oauth. - **Audience misconfiguration** is the most common production failure — the IdP must mint tokens whose `aud` matches what the broker expects. - The native handlers came from **KIP-768** (OIDC-compliant validator + login callback).

  • What happens to OAUTHBEARER authentication if the IdP rotates its signing key and the broker's JWKS cache is stale?
    Tokens signed with the new key carry a new kid the broker can't match, so signature verification fails and auth is rejected until the broker refreshes JWKS (jwks.endpoint.refresh.ms) or a forced refresh occurs on unknown kid.
  • Can the native validator handle opaque (non-JWT) access tokens?
    No. The built-in OAuthBearerValidatorCallbackHandler expects a self-contained JWT. Opaque tokens need IdP introspection (RFC 7662), which requires a custom server callback or a third-party library.

saying these in an interview costs you the question

  • Saying the broker calls the IdP to validate every token — it verifies signatures locally against cached JWKS keys, no per-token introspection.
  • Confusing audience and issuer checks, or forgetting expiry validation.
  • Assuming the principal is always the username — it's whatever claim sub.claim.name names (default sub).
  • Thinking JWKS keys are fetched once and never refreshed.

context

resolving links…