Why does a credential attached to the node grant far more than one issued to a single workload?

seniormust knowfreq 50%

answer

the machine is not the caller
anything on the host can ask
the grant becomes a union
the scheduler keeps widening that union
the far side records the node, not the workload

basics

~20 s

A node credential belongs to the machine, so its grant must cover everything any workload placed there needs, and anything running on that host can reach it. A per-workload token names one workload and carries only that workload's access.

solid answer

~50 s

Two things widen it. First, reach: a credential held by the machine sits where processes on that machine can get at it, so a workload only has to run there to use it — the boundary around each container was never designed to stop that, and platforms that do block the path are mitigating a shared credential rather than replacing it. Second, breadth: the node's grant has to be the union of what every workload the scheduler might place there needs, so the log forwarder on that node can do whatever the reconciliation job can. Placement is not fixed either, so that union grows without anyone editing the grant or re-running the review that approved it. Per-workload identity removes both: each token names one workload, carries only that workload's grant, and shows up in the far side's records as that workload.

Two things widen it. First, **reach**: a credential held by the machine sits where processes on that machine can get at it, so a workload only has to run there to use it — the boundary around each container was never designed to stop that, and platforms that do block the path are mitigating a shared credential rather than replacing it. Second, **breadth**: the node's grant has to be the union of what every workload the scheduler might place there needs, so the log forwarder on that node can do whatever the reconciliation job can. Placement is not fixed either, so that union grows without anyone editing the grant or re-running the review that approved it. Per-workload identity removes both: each token names one workload, carries only that workload's grant, and shows up in the far side's records as that workload.

code

yaml · 13 lines

yaml

workloads:
  - name: nightly-reconciler
    identity: reconciler          # platform mints tokens naming this identity
    tokenAudience: ledger-store
    tokenLifetime: 10m

  - name: log-forwarder
    identity: log-forwarder
    tokenAudience: archive-store
    tokenLifetime: 10m

# Both may be placed on the same host. Neither can read the other's token,
# and neither grant covers the other's audience.

go deeper

for a junior

Recall the core contrast: a credential on the machine can be used by anything running there, while a token issued to a workload can be used only by that workload and only for what it was granted.

for a middle

Explain both halves — reach and breadth — and why the union grant grows on its own. The scheduler widening an approved grant without anyone editing it is the part most candidates miss.

for a senior

Bring the incident view: name what the far side's records show in each case, what containment looks like when the credential is shared, and why draining the node is the only lever you have left.

for a principal

Set the standard: which credential the machine is allowed to hold at all, how the two grant sets are kept disjoint, and how you detect a workload quietly riding on the node's credential instead of owning an identity.

Two places a credential can live

When a workload has to reach a service outside the cluster, the credential it uses is attached to one of two things: the machine it happens to be running on, or the workload itself. Everything else about this comparison follows from that one choice.

A machine-scoped credential is held by the host and handed to whatever asks for it there. A workload-scoped credential is minted by the platform for one declared identity, addressed to one audience, and delivered only into that workload.

Reach: who can obtain it

A credential the host holds is reachable from the host. A container's boundary gives a workload its own view of the filesystem, process table and network stack, but the host's credential is typically reachable over a local address or a path that the platform exposes, and a workload that can reach it can use it. Two consequences follow:

getting a workload scheduled onto the node is enough to obtain the credential — no exploit of the boundary is required;
the credential is equally available to the least-defended workload on the node as to the most carefully reviewed one.

Platforms can and do block that path — filtering the local address, or refusing to expose the credential to tenant workloads at all. That is worth doing, and it is a mitigation of a shared credential, not the same thing as a credential that names the caller. Assuming the path is closed by default is where most of these findings start.

Breadth: whose grant it carries

A credential that names the machine cannot distinguish between the workloads on it, so its grant has to be the union of what all of them need. Put a reconciliation job that reads a ledger store and a log forwarder that writes an archive on the same node, and the node's grant must allow both — which means the log forwarder can read the ledger, and the reconciler can write the archive. Nobody decided that; it fell out of placement.

And placement moves. The scheduler puts workloads where capacity is, replaces them elsewhere when a node fails, and adds new ones over time. A grant that was proportionate when the node ran one workload widens every time the scheduler makes a decision, with no edit to the grant and no re-review.

Per-node against per-workload

	credential attached to the node	token issued per workload
who can obtain it	anything running on that host that can reach it	the one workload it was minted into
whose grant it carries	the union of every workload placed there	that workload's grant alone
what the far side records	the machine	the workload's identity name
effect of a scheduling change	the shared grant quietly covers more workloads	none; each workload brings its own
typical lifetime	long, rotated on a human schedule	minutes, refreshed by construction
one workload compromised	the whole union is available	that workload's grant, for its window

What this changes in an incident

With a machine-scoped credential, the question which workload read this data has no answer, because the far side's records name the host. Containment is also coarse: you cannot withdraw access from the compromised workload without withdrawing it from everything else on that node, so the practical response is to drain the node and hope the workload does not carry its problem to the next one.

With per-workload identity, both answers exist. The records name the identity, so the blast radius is a set you can enumerate rather than infer. Withdrawing the grant for that one identity stops the flow at the far side within one token lifetime, and every other workload on the node is unaffected.

The node still needs a credential

This is not an argument that a machine holds nothing. The node agent has its own work — registering the node, pulling images, reporting state — and it needs a credential for that. The rule is that the two grant sets stay disjoint:

the machine's credential grants only what the node agent's own job requires;
nothing a tenant workload needs is reachable through it;
any workload found using the machine's credential is treated as an unowned grant and given an identity of its own.

The short version an interviewer is listening for: a node credential answers where the call came from; a workload token answers who made it — and only the second one is a boundary you can reason about.

## Two places a credential can live When a workload has to reach a service outside the cluster, the credential it uses is attached to one of two things: **the machine it happens to be running on**, or **the workload itself**. Everything else about this comparison follows from that one choice. A machine-scoped credential is held by the host and handed to whatever asks for it there. A workload-scoped credential is minted by the platform for one declared identity, addressed to one audience, and delivered only into that workload. ## Reach: who can obtain it A credential the host holds is reachable from the host. A container's boundary gives a workload its own view of the filesystem, process table and network stack, but the host's credential is typically reachable over a local address or a path that the platform exposes, and a workload that can reach it can use it. Two consequences follow: - getting a workload scheduled onto the node is enough to obtain the credential — no exploit of the boundary is required; - the credential is equally available to the least-defended workload on the node as to the most carefully reviewed one. Platforms can and do block that path — filtering the local address, or refusing to expose the credential to tenant workloads at all. That is worth doing, and it is a **mitigation of a shared credential**, not the same thing as a credential that names the caller. Assuming the path is closed by default is where most of these findings start. ## Breadth: whose grant it carries A credential that names the machine cannot distinguish between the workloads on it, so its grant has to be the **union** of what all of them need. Put a reconciliation job that reads a ledger store and a log forwarder that writes an archive on the same node, and the node's grant must allow both — which means the log forwarder can read the ledger, and the reconciler can write the archive. Nobody decided that; it fell out of placement. And placement moves. The scheduler puts workloads where capacity is, replaces them elsewhere when a node fails, and adds new ones over time. **A grant that was proportionate when the node ran one workload widens every time the scheduler makes a decision**, with no edit to the grant and no re-review. ## Per-node against per-workload | | credential attached to the node | token issued per workload | |---|---|---| | who can obtain it | anything running on that host that can reach it | the one workload it was minted into | | whose grant it carries | the union of every workload placed there | that workload's grant alone | | what the far side records | the machine | the workload's identity name | | effect of a scheduling change | the shared grant quietly covers more workloads | none; each workload brings its own | | typical lifetime | long, rotated on a human schedule | minutes, refreshed by construction | | one workload compromised | the whole union is available | that workload's grant, for its window | ## What this changes in an incident With a machine-scoped credential, the question *which workload read this data* has no answer, because the far side's records name the host. Containment is also coarse: you cannot withdraw access from the compromised workload without withdrawing it from everything else on that node, so the practical response is to drain the node and hope the workload does not carry its problem to the next one. With per-workload identity, both answers exist. The records name the identity, so the blast radius is a set you can enumerate rather than infer. Withdrawing the grant for that one identity stops the flow at the far side within one token lifetime, and every other workload on the node is unaffected. ## The node still needs a credential This is not an argument that a machine holds nothing. The node agent has its own work — registering the node, pulling images, reporting state — and it needs a credential for that. The rule is that the two grant sets stay **disjoint**: 1. the machine's credential grants only what the node agent's own job requires; 2. nothing a tenant workload needs is reachable through it; 3. any workload found using the machine's credential is treated as an unowned grant and given an identity of its own. The short version an interviewer is listening for: *a node credential answers where the call came from; a workload token answers who made it — and only the second one is a boundary you can reason about.*

What does per-workload identity change about the far service's records?
The caller stops being a machine and becomes a name you chose. Records answer 'which workload read this' directly, so a compromise gives you an enumerable blast radius instead of a guess, and an unused grant becomes visible because nothing ever presents that identity.
The platform blocks workloads from reaching the host's credential — is that equivalent?
No. It removes the reach problem and leaves the breadth problem: the credential is still one grant covering every workload the node serves, still long-lived, and still records the machine rather than the caller. It is a good mitigation and a poor substitute.

saying these in an interview costs you the question

Says the node's credential is fine because only trusted workloads land there
Assumes a workload cannot reach a credential it was not handed
Thinks the set of workloads on a node stays fixed after review
Believes narrowing the node's grant is equivalent to per-workload identity
Says the far side's records show which workload made the call
Treats per-workload identity as a reporting nicety rather than a boundary

Why does a credential attached to the node grant far more than one issued to a single workload?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~4 min

Two places a credential can live

Reach: who can obtain it

Breadth: whose grant it carries

Per-node against per-workload

What this changes in an incident

The node still needs a credential

▾common follow-ups2 questions

▾red flags6 flags

Two places a credential can live

Reach: who can obtain it

Breadth: whose grant it carries

Per-node against per-workload

What this changes in an incident

The node still needs a credential

answer

go deeper