skip to content

A workload moved to a non-root user now fails at start-up reading its mounted config file — what about the delivery shape explains it?

level: seniorimportance: should knowfreq 34%

answer

  1. one shape has a permission check
  2. owner and mode against the running identity
  3. every directory on the path, not just the file
  4. read-only tree is a separate refusal
  5. fix the declaration, not the identity

basics

~20 s

A mounted file has an owner and a mode, so reading it is a permission check. The projected tree was created owned by an identity the workload no longer runs as, and a restrictive mode then denies the open. An inherited variable carries no permissions at all.

solid answer

~40 s

Delivering configuration as files means every read is subject to the filesystem's rules: the file's **owner**, its **mode**, and the traverse permission on every directory on the path. When a workload is switched from a privileged identity to a specific non-root one, a tree that was created owned by the old identity with owner-only permissions becomes unreadable, and the process fails on the open rather than on anything to do with its own logic. The fix belongs in the delivery declaration — set the projected tree's owner and mode to match the identity the workload runs as — not in making the workload privileged again. Note the symmetric hazard: a permissive mode makes the file readable by every process in the container, which is exactly the property you chose a file to avoid.

go deeper

for a junior

Know that a mounted configuration file is a real file with an owner and a mode, so a process running as a non-root user can be refused when it tries to open it. An inherited variable has no such check.

for a middle

Explain what the open actually checks: traverse permission on each directory in the path, then the file's owner and group, then the mode — and that a read-only tree refuses writes independently of all of that.

for a senior

Walk the triage in order and say what each step rules out, then fix it in the delivery declaration rather than by restoring a privileged identity. Name the opposite hazard too: a permissive mode that fails silently.

for a principal

Treat identity and delivery as one change: when a workload's running identity moves, the projected tree's owner and mode move with it. Set the default narrow and make a refused open report the path, so this costs minutes instead of an incident.

## Why this failure exists at all in one shape and not the other The two delivery shapes differ in one respect that only shows up under a non-root identity: **a file is subject to a permission check and an inherited variable is not.** An inherited value is already in the process's memory before its first instruction runs; there is nothing to open and no identity to check. A mounted file must be found, traversed to, and opened, and each of those steps can be refused. Choosing the file shape buys access control, and this failure is the bill for it. ## The two directions this goes wrong Both are common, and they pull against each other: - **Too strict.** The projected tree is created owned by one identity with owner-only permissions, and the workload runs as a different one. The open is refused and the process exits during start-up — often before any of its own logging is configured, which is why the symptom is usually a bare start-up failure rather than a useful message. - **Too permissive.** The tree is created world-readable so that anything in the container can read it. Nothing fails, and the property you chose a file for — a narrower reader set than the inherited environment — is quietly gone. Platforms set a default mode for projected files, and where that default is permissive it is chosen to avoid the first failure, not to protect the contents. ## What is actually being checked The open is not one check. Walk them in order: 1. **Every directory on the path** must be traversable by the identity the process runs as. A correct mode on the file behind an unreadable parent directory still fails. 2. **The file's owner and group** decide which of the mode's permission sets applies to this identity. 3. **The mode** decides whether that set includes read. 4. **Write, separately.** A configuration tree mounted read-only refuses a write regardless of the mode, and a process that expects to rewrite its own configuration file fails there instead. ## Diagnosing it The reasoning chain is short, and it is worth doing in this order because each step rules something out: 1. **Distinguish "not there" from "there but unreadable".** These are different errors with different causes; a missing path means the delivery did not happen at all, while a refusal means it did. 2. **Establish the identity the process actually runs as**, which is not always the one the image declares — the workload spec can override it. 3. **Compare that identity against the owner, group and mode of the file and of every directory above it.** 4. **Check the read-only question separately** if the failure is on a write rather than an open. | symptom | what it points at | |---|---| | the path does not exist | the source was never mounted, or the path is wrong | | the open is refused | owner, group or mode against the running identity | | the directory listing is refused | traverse permission on a parent directory | | the write is refused but reads work | the tree is mounted read-only | ## The fix, and the wrong fix The fix belongs in the **delivery declaration**: the workload spec has a way to state the owner and the mode of the projected tree, and it should be set to the identity the workload runs as, with the narrowest mode that lets it read. Make the delivery match the workload's identity — not the other way around. The wrong fix is to put the workload back on a privileged identity so the permission check stops mattering. It resolves the symptom and gives up a property that was worth far more than the twenty minutes it saves. The second wrong fix is to open the mode up to everything, which silently converts the file back into something with the same reader set as an inherited variable, except now nobody expects that. ## What this says about the two shapes This failure is a feature reporting for duty. The inherited set has no permission failure mode because it has no permissions — anything in the container that can start a process gets the values. A file can be denied, which means it can also be *restricted*, and the cost of that is one more thing that has to be declared correctly and one more class of start-up failure to recognise. When a workload's identity changes, the delivery declaration is part of the change, and forgetting it is how this lands in production.

  • Why is the symptom usually an unhelpful start-up failure rather than a clear message?
    Because the read happens before the process has configured its own logging — the configuration it needs to set that up is the thing it could not read. A workload that resolves its delivery paths first, checks them, and reports which path was refused, turns a five-minute triage into a ten-second one.
  • What is the equivalent failure for a value delivered as an inherited variable?
    There isn't one, and that is the trade. A variable is in memory before the first instruction, so it cannot be refused — which also means it cannot be restricted. The absence of this failure mode and the absence of access control are the same property viewed from two sides.
  • Why is a permissive default mode on a projected tree the more dangerous of the two directions?
    Because it does not fail. A too-strict mode stops the workload at start-up and gets fixed within the hour; a too-permissive one ships, and the file's narrower reader set — the reason the file shape was chosen — is gone with nothing to announce it.

saying these in an interview costs you the question

  • Running the workload as root again is the fix
  • A mounted file ignores ownership once it is inside a container
  • Only the file's mode matters, not the directories above it
  • A permission failure and a missing path are the same problem
  • A world-readable config file is harmless inside a container
  • The identity in the image is always the one the process runs as