For a multi-tenant job platform that runs short-lived jobs against Kafka, how would you choose between OAUTHBEARER, delegation tokens, and per-tenant credentials for worker/job impersonation?
answer
- OAUTHBEARER = central identity, per-job JWT, token exchange for on-behalf-of
- delegation tokens = coordinator mints, executors fan out over SCRAM, no IdP per task
- revocation: short lifetimes + connections.max.reauth.ms; deleg adds --expire
- blast radius: many short-lived narrow tokens > few long-lived secrets
- static SCRAM/mTLS = simple but rotates poorly
basics
~20 sUse OAUTHBEARER when each job/worker can talk to your IdP for its own short-lived token (best for centralized identity and revocation). Use delegation tokens when a coordinator authenticates once and must fan out cheap, revocable credentials to many executors without contacting the IdP per task.
solid answer
~50 sThe decision hinges on who holds identity and how workers get credentials. OAUTHBEARER centralizes identity in the IdP: each job acquires a short-lived JWT via client-credentials (or token exchange / on-behalf-of for true impersonation), and the broker validates it statelessly against JWKS. This gives clean per-tenant principals, central revocation policy, and fine-grained scopes — but every worker needs IdP reachability and a client secret, and revocation only bites on new connections or re-auth (`connections.max.reauth.ms`). Delegation tokens (KIP-48) suit a coordinator/executor topology: the driver authenticates once (possibly via OAUTHBEARER), mints SCRAM-backed tokens, and distributes them to executors that never touch the IdP — cheaper at fan-out and individually revocable, but bounded by `delegation.token.max.lifetime.ms` and dependent on a shared broker secret. Per-tenant static SCRAM/mTLS credentials are simplest but rotate poorly and leak widely. For short-lived multi-tenant jobs I'd default to OAUTHBEARER with token exchange for impersonation, and reach for delegation tokens where IdP fan-out or air-gapped executors make per-task token acquisition impractical.
go deeper
Recognize there are different ways jobs get Kafka credentials and tokens are safer than shared passwords.
Contrast IdP-issued OAUTHBEARER tokens with broker-issued delegation tokens at a high level.
Weigh revocation, blast radius, and IdP dependency to pick between mechanisms for a given topology.
Architect a multi-tenant impersonation model: token exchange for on-behalf-of, delegation tokens for fan-out, reauth-bounded revocation, and per-tenant auditability.
## Framing the decision "Impersonation" here means a worker/job acting as (or on behalf of) a tenant identity when talking to Kafka, with authorization (ACLs) applied to that identity. Three credential strategies: ### 1. OAUTHBEARER (per-job IdP tokens) - **How**: each job acquires a short-lived JWT from the IdP (`OAuthBearerLoginCallbackHandler`, client-credentials grant). For true *on-behalf-of* impersonation, use OAuth2 **token exchange (RFC 8693)** or the IdP's on-behalf-of flow so the worker's token carries the tenant's subject/scopes. - **Pros**: centralized identity, central revocation/rotation, fine-grained scopes, stateless broker validation via JWKS (KIP-768), clean per-tenant principals for ACLs. - **Cons**: every worker needs IdP network reachability and a client secret to bootstrap; IdP becomes a hot path and a dependency; token expiry only enforced on live connections via `connections.max.reauth.ms`; opaque tokens unsupported by the native validator. ### 2. Delegation tokens (KIP-48) - **How**: a coordinator authenticates once (often via OAUTHBEARER), calls `kafka-delegation-tokens.sh --create`, and distributes tokenId/HMAC to executors that authenticate over SCRAM. - **Pros**: cheap fan-out (no IdP call per task), executors need no IdP access or client secret, individually revocable (`--expire`), short-lived, renewable up to a cap. - **Cons**: capped by `delegation.token.max.lifetime.ms` (default 7d); requires a shared `delegation.token.secret.key` across brokers; the token's principal is the *owner*, so per-tenant separation requires per-tenant coordinators/owners; bearer-style HMAC needs TLS. ### 3. Static per-tenant SCRAM / mTLS - **Pros**: dead simple, no IdP runtime dependency. - **Cons**: long-lived secrets distributed widely, painful rotation, weak revocation, large blast radius on leak — poor fit for many short-lived multi-tenant jobs. ## A decision rubric - **Strong central identity + per-tenant ACLs + acceptable IdP reachability** -> OAUTHBEARER (with token exchange for on-behalf-of). - **Coordinator/executor fan-out, air-gapped or IdP-throttled executors, short jobs** -> delegation tokens, bootstrapped by a coordinator that itself uses OAUTHBEARER/Kerberos. - **Small, static tenant set, no IdP** -> per-tenant SCRAM, accepting rotation cost. ## Cross-cutting concerns - **Revocation latency**: OAUTHBEARER and delegation tokens both rely on short lifetimes; for live-connection enforcement set `connections.max.reauth.ms`. Delegation tokens add immediate `--expire`. - **Blast radius**: prefer many narrowly-scoped short-lived tokens over a few broad long-lived secrets. - **Operational coupling**: OAUTHBEARER couples Kafka availability to IdP availability at connect/re-auth time; delegation tokens couple it to the broker secret-key management and metadata replication (KRaft). - **Auditing**: per-job tokens with distinct subjects give cleaner audit trails than a shared service credential. ## The pragmatic answer Most modern platforms standardize on **OAUTHBEARER with token exchange** for clean per-tenant impersonation and central control, and keep **delegation tokens** as the tool for high-fan-out or IdP-isolated execution. Static credentials are a fallback for the simplest cases.
- Which OAuth2 flow gives true on-behalf-of impersonation so a worker's token carries the tenant's identity, not the worker's?OAuth2 token exchange (RFC 8693) or the IdP's on-behalf-of flow, which issues a token whose subject/scopes represent the tenant while the worker acts as the actor — the broker then applies ACLs to the tenant principal.
- Why might delegation tokens be preferable even in an OAUTHBEARER shop?When a coordinator must fan credentials out to many short-lived executors that lack IdP reachability or where per-task token acquisition would overload the IdP; the coordinator authenticates once (via OAUTHBEARER) and issues cheap, individually-revocable SCRAM tokens.
saying these in an interview costs you the question
- Treating delegation tokens and OAUTHBEARER as interchangeable rather than complementary.
- Assuming either gives instant revocation on live connections without connections.max.reauth.ms.
- Proposing to distribute the IdP client secret or a keytab to every executor.
- Ignoring that a delegation token's principal is its owner, so per-tenant separation needs per-tenant ownership.
- Recommending static long-lived per-tenant secrets for large short-lived multi-tenant workloads.