skip to content

When `docker run -e` gives way to secret files a scheduler mounts, what must the image change?

level: seniorimportance: should knowfreq 38%

answer

  1. The environment is not a private channel
  2. docker inspect prints Config.Env
  3. Read a path, not a value
  4. The _FILE convention with a fallback
  5. Mount mode must match the declared USER

basics

~20 s

The image must read the credential from a file path given by configuration rather than from an environment variable, tolerate the file changing under it, and be able to read it as its non-root USER. Environment values are exposed by docker inspect and inherited by every child process.

solid answer

~50 s

Environment variables were never a good secret channel: `docker inspect` prints them from `Config.Env`, every child process inherits them, and they appear in `/proc/<pid>/environ` and in crash dumps. A scheduler mounts secrets as read-only files instead — Docker's own Compose and Swarm surface them under `/run/secrets/<name>` — which keeps the value out of the container's config and lets it be rotated without rebuilding the image. To accept that, the image should read a **path** from configuration, following the common `*_FILE` convention so a variable such as `CHECKOUT_DB_PASSWORD_FILE` points at the file while `CHECKOUT_DB_PASSWORD` stays as a fallback for local runs. Two practical traps: the mounted file's mode and owner must let your declared non-root `USER` read it, and a process that reads the secret once at startup keeps a stale credential after rotation. Build-time credentials are a separate mechanism — `RUN --mount=type=secret`.

code

bash · 7 lines
bash
#!/bin/sh
set -eu
if [ -n "${CHECKOUT_DB_PASSWORD_FILE:-}" ] && [ -r "$CHECKOUT_DB_PASSWORD_FILE" ]; then
  CHECKOUT_DB_PASSWORD="$(tr -d '\n' < "$CHECKOUT_DB_PASSWORD_FILE")"
  export CHECKOUT_DB_PASSWORD
fi
exec /checkout "$@"

go deeper

for a junior

Know that anything passed with -e is visible in docker inspect and inherited by child processes, and that a secret must never be written into a Dockerfile with ENV or COPY.

for a middle

Explain the mechanics of both delivery paths: what a mounted secret file looks like inside the container, why /run/secrets is a memory-backed read-only mount, and how the *_FILE convention lets one image serve both local runs and a platform.

for a senior

Show that you have hit the real failures — the uid that cannot read the mount, the trailing newline, the entrypoint that logs its configuration, and the rotation that silently required a fleet restart. Say how you would verify each before the migration.

for a principal

Own the standard across teams: one convention for how images accept credentials, a rule that build-time and run-time secrets use different mechanisms, and a clear statement of what rotation costs so that credential lifetime is a decision rather than an accident.

## Why the environment variable was always the weak option `docker run -e CHECKOUT_DB_PASSWORD=hunter2` looks harmless and is the most common way credentials reach a container, but the value is stored in the container's configuration and leaks in several ordinary directions: - `docker inspect checkout` prints it under `Config.Env`. Anyone with access to the daemon socket reads it, and so does every tool that dumps container config for debugging or inventory. - Every child process inherits the whole environment. A build script, a shell you `docker exec` into, a subprocess that shells out — all of them carry the secret, and any of them may log its own environment on error. - `/proc/<pid>/environ` exposes it to anything sharing the PID namespace, and process crash dumps and error reporters routinely capture the environment. - It is easy to promote accidentally into an image: an `ENV` line, or a `--build-arg` that ends up recorded in build history. None of this is catastrophic on a laptop. It becomes material once the workload runs on shared infrastructure that many people can inspect. ## What the scheduler does instead Schedulers deliver secrets as **files**: a small read-only mount, usually backed by memory rather than disk, whose contents are the raw secret value. Docker's own Compose and Swarm follow the same shape, placing each secret at `/run/secrets/<name>`; other platforms let you choose the mount path. The advantages are concrete rather than cosmetic. The value is not in the container's config, so `docker inspect` does not print it. It is not inherited by child processes. It can be rotated by replacing the file's contents, without touching the image or the container's environment. And it can be scoped: this workload gets this secret, rather than the whole environment block being handed to every process in the container. ## The changes the image must make **Read a path, not a value.** The durable convention is a `*_FILE` variable alongside the plain one. For an order-checkout API: ``` CHECKOUT_DB_PASSWORD_FILE=/run/secrets/checkout_db_password ``` The startup code prefers the `_FILE` variable when set, reads the file, trims the trailing newline that almost every secret store adds, and falls back to the plain variable so a developer's `docker run` still works. Supporting both is what lets one image run everywhere, which is the whole point of the handoff. **Make the file readable by your `USER`.** This is the failure that shows up an hour into the migration. The image declares `USER 10001:10001`; the platform mounts the secret owned by root with mode `0400`; the process gets `EACCES` at startup and the instance crash-loops with a message that says nothing about permissions. The mount's mode and owner are runtime configuration — set them to match the uid the image declares, and check with `docker exec` (or an equivalent shell in the platform) that the file is actually readable as that user, not as root. **Decide what rotation means.** Reading the secret once at startup is legitimate, but then a rotation only takes effect on the next restart, and you have quietly made "rotate the credential" mean "restart the fleet". The alternatives are to re-read the file when authentication fails, or to watch it and refresh a cached credential. Pick one deliberately and write it down, because the failure mode of the wrong choice is a fleet that authenticates fine until the day the old credential is revoked. **Never log it, never echo it.** An entrypoint that prints its configuration on startup for debugging will print the secret straight into the log stream the platform collects. ## Build-time is a different mechanism Do not confuse a runtime secret with a **build** secret. If the build needs a credential — a private package repository, say — the tool is BuildKit's mount: ``` RUN --mount=type=secret,id=npmrc,target=/root/.npmrc \ npm ci --omit=dev ``` The file exists only for the duration of that one `RUN` instruction and is never written into a layer. Passing the same credential as `--build-arg` is the wrong answer: build arguments are visible in the build's history and in whatever logs captured the command, so they are not a secret channel. `COPY`ing a credential in and deleting it in a later instruction is worse — the earlier layer still contains the file even though the final filesystem does not, and anyone who pulls the image can extract it. ## The one-line summary Runtime secret delivery is the scheduler's job; the image's job is to accept a path, read it as an unprivileged user, keep it out of its own logs, and be honest about when it re-reads.

  • The process reads the secret file once at startup. What breaks when the credential is rotated?
    Nothing, until the old credential is revoked — then every instance fails authentication at once, and the fix is a full restart. If you accept read-once, make "rotation requires a rolling restart" an explicit part of the runbook. Otherwise re-read the file when authentication fails, or watch it and refresh the cached value, so rotation is a file replacement rather than a deployment.
  • Why not pass the credential with `docker build --build-arg`?
    Build arguments are not a secret channel: the value is visible in the build's history and in any log or CI transcript that captured the command, and it is trivial to leak it into a layer by referencing it in a `RUN`. BuildKit's `RUN --mount=type=secret,id=...` exposes the file only while that instruction executes and writes nothing into the resulting layer.
  • An image declares `USER 10001` and the platform mounts the secret as root with mode 0400. What happens?
    The process cannot open the file and fails at startup, usually with a permission error that does not name the secret. Mount mode and ownership are runtime configuration, so set them to the uid the image declares — and verify by reading the file as that user inside a running container rather than as root, which will succeed regardless.

saying these in an interview costs you the question

  • Calls environment variables a secure secret channel
  • Passes credentials with docker build --build-arg
  • COPYs a credential in and deletes it in a later layer
  • Ignores the trailing newline in a secret file
  • Mounts the secret unreadable by the container's USER
  • Prints the resolved configuration to stdout at startup

context

resolving links…