skip to content

Compare the old long-lived ServiceAccount token that Kubernetes stored in a Secret with the token a modern cluster projects into a Pod through the TokenRequest API. What changed regarding expiry, audience, and object binding, and why did those changes matter?

level: middleimportance: must knowfreq 50%

answer

  1. legacy: no exp, no aud, no binding, lived in a Secret
  2. TokenRequest: exp ~1h, aud claim, bound to Pod+SA UID
  3. kubelet rewrites the file at ~80% of lifetime
  4. apps must re-read the file, or 401 after an hour
  5. 1.21 default, 1.24 no auto-created Secret, kubectl create token

basics

~20 s

The old token was a non-expiring JWT sitting in a Secret, valid for any audience and any holder forever. The projected token from the TokenRequest API expires (about an hour, refreshed in place by the kubelet), is scoped to a named audience, and is bound to the Pod and ServiceAccount, so it dies when the Pod does.

solid answer

~50 s

The legacy token was a JWT stored in a `kubernetes.io/service-account-token` Secret, auto-created for every ServiceAccount. It had **no expiry**, **no audience restriction**, and **no binding** to anything, so a copy exfiltrated from a container or read out of etcd stayed valid indefinitely and could be replayed anywhere. The modern one is minted through the **TokenRequest** API and delivered as a `projected` volume. It carries an expiry (kubelet default about an hour, and the kubelet rewrites the file at roughly 80% of lifetime), an `aud` claim naming the intended recipient so the API server rejects tokens meant for another service, and `kubernetes.io` claims binding it to the Pod and ServiceAccount UIDs, so the API server rejects it once the Pod is gone. The practical consequence for application code: the token on disk changes, so it must be re-read rather than cached at startup. Official clients do this; hand-rolled code that reads the file once starts getting 401s after about an hour.

code

bash · 11 lines
bash
kubectl create token payments-api -n payments \
  --duration=30m --audience=vault

# inspect the claims of the token a Pod is carrying
kubectl exec payments-api-0 -- \
  cat /var/run/secrets/kubernetes.io/serviceaccount/token \
  | cut -d. -f2 | base64 -d 2>/dev/null | jq
# { "aud": ["https://kubernetes.default.svc"], "exp": ...,
#   "kubernetes.io": { "namespace": "payments",
#                      "pod": { "name": "payments-api-0", "uid": "..." },
#                      "serviceaccount": { "name": "payments-api", "uid": "..." } } }

go deeper

for a junior

Know that the token in the Pod expires and is refreshed by the kubelet, unlike the old permanent one stored in a Secret.

for a middle

Name all three properties (expiry, audience, Pod/ServiceAccount binding), what each prevents, and the re-read requirement for application code.

for a senior

Add the version timeline, the refresh mechanics at roughly 80% of lifetime, kubectl create token for out-of-cluster consumers, and how you find and retire remaining legacy tokens.

for a principal

Position bound tokens as the foundation for federation: short-lived, audience-scoped assertions of workload identity that external systems can verify, removing whole classes of stored credentials from the platform.

## The legacy design and its problem Historically, creating a ServiceAccount caused a controller to create a companion Secret of type `kubernetes.io/service-account-token` containing a signed JWT, and the kubelet mounted that Secret into Pods using the account. The token was a bearer credential with three unfortunate properties: - **No expiry.** The JWT had no `exp` claim. Once minted it was valid until the ServiceAccount or its Secret was deleted, so revocation meant deleting the object and rolling everything using it. - **No audience.** There was no `aud` restriction, so a token intended for the Kubernetes API could be replayed against any other service that had been taught to accept these JWTs. A service that validated the signature but not the intended recipient could be attacked with a token harvested from an unrelated workload (the "confused deputy" shape). - **No binding.** The token asserted only the ServiceAccount. It said nothing about which Pod it was issued for, so it remained valid after the Pod was deleted and was equally usable from an attacker's laptop. Combined with the fact that the token lived in a Secret, this meant that anyone able to read Secrets in a namespace obtained permanent, portable credentials for every workload identity there. ## The TokenRequest design The `TokenRequest` subresource (`POST /api/v1/namespaces/<ns>/serviceaccounts/<name>/token`) mints a token on demand with requested properties, and the `serviceAccountToken` projected-volume source lets the kubelet do that on the Pod's behalf. Since Kubernetes 1.21 this is how Pods get their credential by default (`BoundServiceAccountTokenVolume`), and since 1.24 no Secret-based token is auto-created any more. What the new token carries: - **`exp`** — a real expiry. The kubelet's default projected lifetime is one hour; a manifest can request a different `expirationSeconds` (the cluster enforces a minimum of 10 minutes and a configured maximum). - **`aud`** — the audience or audiences. The API server only accepts tokens whose audience includes its own; a token minted with `--audience=vault` is useless against the API server, and vice versa. This is the property that makes it safe to hand a ServiceAccount token to an external system. - **`kubernetes.io` claims** binding the token to the ServiceAccount UID and, for projected tokens, the Pod name and UID (and optionally the Node). The API server checks that the referenced objects still exist and match, so deleting the Pod invalidates its token immediately. This turns a credential leak from permanent into time-boxed and location-bound. **Refresh.** The kubelet re-requests the token before it expires (at roughly 80% of its lifetime, or when it is older than 24 hours) and rewrites the file atomically. The Pod is never restarted for this. **The application-side consequence.** Because the file changes, code must read it per request or on a timer. Official client libraries (client-go and the language clients built on the same behaviour) reload it automatically. Hand-rolled code that reads `/var/run/secrets/.../token` once into a variable at startup works fine for an hour and then gets `401 Unauthorized` on everything, which is the classic symptom of this migration. The same applies to any client where credentials are cached in a connection pool or an SDK configured once. ## Getting a token manually `kubectl create token <sa>` calls TokenRequest directly and supports `--duration` and `--audience`. This is the supported way to obtain a token for a CI system or for debugging, and it deliberately produces something that expires. If a genuinely long-lived token is unavoidable, you can still create a `kubernetes.io/service-account-token` Secret by hand with the `kubernetes.io/service-account.name` annotation, but this is discouraged: from 1.29 the control plane labels such tokens with their last-used date and a cleanup controller removes ones that have gone unused, precisely so the legacy pattern is visible and decays. ## What to say Name the three changed properties (expiry, audience, object binding), explain the attack each one closes, mention that the kubelet refreshes the file in place so applications must re-read it, and note that `kubectl create token` is the modern way to obtain one by hand.

  • An application reads the token file once at startup. What happens, and when?
    It works until the projected token expires, typically about an hour, then every API call returns 401 Unauthorized while the file on disk holds a perfectly valid newer token. The kubelet refreshed the file in place and the process never noticed. The fix is to re-read the file per request or on a timer, which official client libraries already do.
  • Why does the audience claim matter when you hand a ServiceAccount token to an external system such as Vault or a cloud STS?
    Because it prevents replay across services. A token minted with `audience: vault` is rejected by the Kubernetes API server, and a token for the API server is rejected by a Vault configured to require its own audience. Without that, any service that trusts the cluster's signing key could be attacked with a token harvested from an unrelated workload, and one leaked credential would be usable everywhere.
  • How do you revoke a projected ServiceAccount token that you believe has been exfiltrated?
    The direct lever is that the token is bound to the Pod and ServiceAccount UIDs, so deleting the Pod invalidates it immediately, and deleting and recreating the ServiceAccount invalidates every token issued for it because the UID changes. Beyond that, the short expiry means the exposure window closes on its own, which is the main reason bound tokens replaced the non-expiring ones.

The old token was a house key with no expiry that opened any building that had been told to trust that keyring. The new one is a visitor badge stamped with the building it is for, the person it was issued to, and an expiry time, reprinted quietly before it lapses.

saying these in an interview costs you the question

  • Believing the projected token file is static and can be cached for the life of the process
  • Thinking a ServiceAccount still gets an auto-created Secret containing a token on a current cluster
  • Assuming a token minted for one audience is accepted by the Kubernetes API server
  • Saying tokens cannot be revoked, ignoring that Pod and ServiceAccount UID binding invalidates them
  • Creating long-lived Secret-based tokens for convenience and treating that as the normal path

context