How do you integrate Kafka GSSAPI with enterprise Active Directory, and how do you map Kerberos principals to Kafka users for ACLs?

## AD is a KDC Microsoft **Active Directory** implements Kerberos v5. So integrating Kafka with AD means treating the AD domain as the **Kerberos realm** and the **domain controllers** as the **KDCs**. There is usually no separate MIT KDC to run. ## Setup steps 1. **krb5.conf**: set `default_realm = CORP.EXAMPLE.COM` (realms are conventionally uppercase), and under `[realms]` list `kdc = dc1.corp.example.com` and `admin_server`. Add `[domain_realm]` to map DNS domains (e.g. `.corp.example.com = CORP.EXAMPLE.COM`). 2. **Service accounts + SPNs**: in AD create a user/service account per broker (or a shared one) and register a **Service Principal Name** with `setspn -S kafka/[email protected] <account>`. The SPN's service part must equal `sasl.kerberos.service.name` (kafka). 3. **Keytab export**: use `ktpass` to export a keytab binding the SPN to the account, choosing an encryption type (AES256/AES128). Distribute it securely to the broker. Watch the **kvno** — re-exporting rotates the key version and old keytabs stop working. 4. **Encryption types (etypes)**: AD and the JVM must share an etype. Modern setups use **AES**; legacy **RC4-HMAC** is often disabled for security. Mismatched etypes cause 'KDC has no support for encryption type' errors. The JVM may also need `allow_weak_crypto` settings or, conversely, unlimited-strength crypto enabled (older JDKs). ## Principal-to-Kafka-user mapping After authentication the identity is a full Kerberos principal such as `[email protected]` or `kafka/host@REALM`. ACLs are easier to manage against short names. Kafka maps via: - **`sasl.kerberos.principal.to.local.rules`**: a list of `RULE:[n:string](regex)s/pattern/replacement/` expressions (the same `auth_to_local` grammar from Hadoop/Kerberos) plus `DEFAULT`. Example: `RULE:[1:$1@$0](.*@CORP.EXAMPLE.COM)s/@CORP.EXAMPLE.COM//` turns `[email protected]` into `alice`. The result becomes `User:alice` for ACLs. - **`principal.builder.class`** (a `KafkaPrincipalBuilder`): a custom Java class for logic that rules cannot express — e.g. deriving identity from a certificate for SSL, or attaching group info. ## Authorization with groups Kerberos itself carries only the principal, not AD group membership. To authorize by AD group you either: (a) sync groups into an external authorizer (e.g. a Ranger/OPA-style authorizer or Confluent's LDAP-backed RBAC), or (b) do LDAP lookups in a custom authorizer. Plain Kafka ACLs match the KafkaPrincipal you produced from the mapping rules. ## Cross-realm and trusts If clients live in a different AD domain than the brokers, you need a **cross-realm trust** and `[capaths]` entries so tickets can traverse realms. Mismatched DNS-to-realm mapping (`[domain_realm]`) is a frequent failure point. ## Operational pitfalls - **kvno drift** between AD and the deployed keytab after a password/key reset. - **Clock skew** beyond ~5 min (AD default) -> ticket rejection; keep NTP tight. - **Hostname/FQDN** mismatch between the SPN and how clients address the broker. - **Etype mismatch** between AD policy and JVM/krb5.conf. - **Case sensitivity**: realms uppercase, and auth_to_local rules are order-sensitive — first match wins.