skip to content

You own human access to a 38-node Kubernetes cluster shared by several teams. How do you choose between client certificates and OIDC, and what break-glass path do you keep?

level: principalimportance: should knowfreq 38%

answer

  1. joiner, mover, leaver
  2. groups in the directory
  3. one kubeconfig, one plugin
  4. provider outage needs a way in
  5. break-glass must be loud

basics

~20 s

Use an OIDC identity provider for daily access, with prefixed groups bound by RBAC and short-lived tokens, because leavers and movers are handled centrally. Keep an independent, short-lived or sealed client certificate as an alerted break-glass path.

solid answer

~40 s

I compare them on joiner, mover and leaver handling, credential lifetime and dependencies. Certificates need an approval per person, freeze groups until expiry and can't be revoked, but depend only on the API server. OIDC puts identity and groups in the identity provider, gives minutes-long tokens and one kubeconfig with an `exec` plugin, but makes the provider a dependency. So for a shared cluster I configure an `AuthenticationConfiguration` with a groups claim and an `oidc:` prefix, bind groups rather than people, and use `kubectl auth whoami` in onboarding. For break-glass I keep a certificate path that needs nothing else: minted on demand with a small `expirationSeconds` or sealed offline, bound to a named user so it can be un-bound, and alerted on every use.

go deeper

for a junior

Know that people can log in with certificates or an identity provider, and that a shared admin file is a bad idea.

for a middle

Explain how each option handles group membership and expiry, and what the kubeconfig looks like for each.

for a senior

Show the operational details: prefixed group bindings, token lifetime as revocation delay, verification with kubectl auth whoami, and a tested emergency certificate.

for a principal

Own the tradeoffs: identity-provider availability versus control, token lifetime versus friction, who governs groups, and a parallel-run migration plan with alerting on break-glass use.

## The decision in one sentence For a shared cluster, **everyday human access usually belongs to an OIDC identity provider**, **certificates belong to break-glass**, and the design work is in governing groups, token lifetimes and the path you use when the identity provider is down. There is no single right answer; the constraints below decide it. ## What each option really costs | Concern | Client certificates via the CSR API | OIDC tokens via an exec plugin | |---|---|---| | Joiner | Someone approves a CSR per person | Add the person to an identity-provider group | | Mover | Groups are frozen in the certificate; reissue | Change group membership; applies on the next token | | Leaver | No revocation; edit bindings, maybe rotate the CA | Disable the account; access ends when the current token expires | | Credential lifetime | One year by default unless `expirationSeconds` is set | Minutes, set by the identity provider | | Dependency | None beyond the API server | Identity provider must be reachable for new logins | | Audit identity | Whatever CN was typed at issuance | Stable claim from a managed directory | | Setup | Built in, nothing to operate | API server authentication config, a plugin on every laptop | On a **managed Kubernetes service**, the provider may already map its own cloud identities into the cluster; the same reasoning applies, but the mapping mechanism is the provider's. ## A reference design For a 38-node cluster with a mixed spot and on-demand pool, shared by the fraud-rules engine team and others: 1. **Identity provider as the source of truth.** Configure the API server with an `AuthenticationConfiguration` (`--authentication-config`) that trusts the company issuer, maps a stable username claim, and maps a groups claim with a prefix such as `oidc:`. 2. **Groups, not people, in bindings.** RoleBindings name `oidc:fraud-rules-dev` and similar; group ownership is assigned to team leads in the identity provider, so access reviews happen there. 3. **One distributed kubeconfig.** It carries only the cluster entry and an `exec` block for the credential plugin, so nothing secret is ever copied between people. 4. **Short tokens.** An ID token lifetime of a few minutes bounds how long a disabled account keeps working. 5. **Verification.** `kubectl auth whoami` is part of onboarding, so people see the exact username and groups they arrive with. ## Break-glass: the path you hope not to use The identity provider is now a dependency of every human action. If it is down during an incident, for example halfway through a 13-minute node drain after a wave of spot interruptions, engineers cannot get new tokens. A break-glass path should be: - **Independent**: a client certificate signed by the cluster CA, which needs nothing but the API server. - **Short-lived or sealed**: either minted on demand through the CSR API with a small `expirationSeconds`, or a pre-issued credential kept offline under dual control. - **Narrow where possible**: bound to a named user with a specific ClusterRole rather than a group, so it can be un-bound. - **Loud**: every use triggers an alert from the audit log and a follow-up review. kubeadm clusters already illustrate the split: `admin.conf` authenticates as a member of `kubeadm:cluster-admins`, a group given its power through a ClusterRoleBinding that can be removed, while `super-admin.conf` carries `system:masters`, which bypasses RBAC and cannot be revoked short of CA rotation. The latter should be locked away, not used day to day. ## Tradeoffs to argue explicitly - **Availability versus control.** A second issuer in the authentication config, or a certificate break-glass path, reduces the outage risk but adds another credential surface to govern. - **Token lifetime versus friction.** Shorter tokens mean faster revocation but more frequent plugin refreshes and more load on the identity provider. - **Central groups versus team autonomy.** Letting teams manage their own identity-provider groups speeds onboarding but moves an authorization decision outside the platform team's review. - **Migration.** Moving from certificates to OIDC is safe to run in parallel: both authenticators are active, so bind the new `oidc:` groups first, move people, then remove the certificate-user bindings and let the old certificates expire. ## Signals of a weak design - A single shared admin kubeconfig. - Personal certificates carrying team groups, with no expiry policy. - No tested break-glass path, or one that nobody is alerted about. - Bindings that name individual email addresses scattered across namespaces.

  • Why bind break-glass access to a named user rather than to system:masters?
    A named user is authorized through a ClusterRoleBinding, so you can remove that binding the moment the credential is suspected of leaking. `system:masters` bypasses authorization entirely, so a leaked certificate carrying it stays all-powerful until the trusted CA is rotated. That is why kubeadm separates `admin.conf`, whose group is bound by an ordinary ClusterRoleBinding, from `super-admin.conf`.
  • How would you migrate an existing certificate-based team to OIDC without an outage?
    Run both authenticators together, since the API server accepts either credential. Add OIDC with prefixed groups, create bindings for those groups mirroring the current access, ship the plugin-based kubeconfig, and have people confirm with `kubectl auth whoami`. Then remove bindings for certificate users and groups, and let the old certificates expire or rotate the CA if some cannot be neutralized.

saying these in an interview costs you the question

  • OIDC removes the need for any break-glass credential.
  • Client certificates are more secure because they never expire.
  • Binding individual email addresses is simpler to govern than groups.
  • The day-to-day admin kubeconfig should carry system:masters.
  • Certificates and OIDC cannot be enabled on the same API server.