How would you run the same Docker Compose stack with different settings for local development and for an automated integration-test run, without duplicating the whole file — and how do the `.env` file, the `environment:` key and shell variables interact?
answer
- base + compose.override.yaml auto-merged
- -f order: later wins; explicit -f disables auto-override
- scalars replace, lists append, maps merge
- profiles = whole services on/off
- ${VAR} from shell then .env; environment: is inside the container
basics
~20 sKeep one base file and layer overrides: compose.override.yaml is merged automatically, or pass -f base.yaml -f ci.yaml explicitly. Use profiles to switch optional services on. .env supplies values for ${VAR} interpolation in the file; environment:/env_file: set variables inside the container.
solid answer
~50 s**Layering.** One `compose.yaml` holds what is always true. `compose.override.yaml` is picked up automatically and is the dev-only layer (bind mounts, debug ports). For CI, name the layers explicitly: `docker compose -f compose.yaml -f compose.ci.yaml up --wait`. Merge rules matter: scalars are replaced, most lists are appended (ports, volumes), maps are merged key by key. `!reset` and `!override` tags let a layer clear an inherited value. **Profiles.** `profiles: [tools]` keeps optional services out of a default `up`; `--profile tools` or `COMPOSE_PROFILES` enables them. **Two different variable worlds.** `${VAR}` in the YAML is *interpolated by Compose at parse time*, resolved from the shell environment first, then `.env` next to the file. `environment:`/`env_file:` set variables *inside the container* and are unrelated to interpolation — `env_file` values are not available for `${...}`. Always verify with `docker compose config`, and set a distinct project name per CI job so parallel runs don't collide.
code
yaml · 24 lines# compose.yaml
services:
api:
image: myapp/api:${API_TAG:-dev}
environment:
LOG_LEVEL: info
depends_on:
db: { condition: service_healthy }
db:
image: postgres:16
healthcheck:
test: ["CMD-SHELL", "pg_isready -U postgres"]
interval: 5s
retries: 10
seed:
image: myapp/seed:${API_TAG:-dev}
profiles: ["test"]
# compose.ci.yaml
services:
api:
environment:
LOG_LEVEL: warn # map merge: replaces just this key
ports: !override [] # drop published ports inherited from the basego deeper
Know that compose.override.yaml exists and is merged automatically, and that ${VAR} comes from .env or the shell.
Use -f layering and profiles deliberately, and articulate the difference between interpolation and container environment.
Own the merge semantics (append vs replace, !override/!reset), the precedence order, and CI hygiene: unique project name, --wait, guaranteed down -v.
Set the convention for how many variants exist at all, keep the base file the single contract dev and CI share, and rule on where secrets enter the pipeline rather than accumulating per-environment files.
## The goal: one source of truth, thin variants Copying `compose.yaml` into `compose.ci.yaml` guarantees drift — the copies diverge in the image tag, the healthcheck, the volume name, and the CI stack silently stops testing the dev stack. Compose gives three composable mechanisms instead: file merging, profiles, and interpolation. ## File merging With no `-f`, Compose loads `compose.yaml` and, if present, **`compose.override.yaml` automatically**, merging the second onto the first. That is the intended split: base = the contract (images, service names, dependency graph); override = the local developer conveniences (bind-mounted source, published debug ports, `DEBUG=true`). For other variants, list files explicitly, left to right, later wins: ```bash docker compose -f compose.yaml -f compose.ci.yaml up -d --wait ``` (The `COMPOSE_FILE` env var, with `COMPOSE_PATH_SEPARATOR`, does the same without repeating flags.) Note that once you pass `-f` explicitly, the automatic override file is **not** loaded — a frequent surprise when CI behaves differently from a bare `up`. Merge semantics you should be able to state: - **Scalars** (`image`, `restart`, `user`, `command`) — the later file replaces the earlier. - **Sequences** (`ports`, `volumes`, `expose`, `dns`) — appended, not replaced. This is why an override cannot "remove" a published port simply by declaring a shorter list; you get both. - **Mappings** (`environment`, `labels`, `build.args` when written in map form) — merged per key, later wins per key. - **`!reset`** sets a value back to empty/absent, and **`!override`** replaces a list instead of appending — the escape hatches for the two rules above. Because of the append rule, design the base file to contain the *smallest* common core and put anything variant-specific only in the layers. ## `extends` and fragments Within one file, YAML anchors/aliases (`x-common: &common` … `<<: *common`) and top-level `x-` extension fields reduce repetition. Across files, `extends:` pulls a single service definition from another file — useful for sharing a service between two independent stacks where merging whole files is wrong. ## Profiles Profiles switch **whole services** on and off rather than changing their settings: ```yaml services: app: {} seed: profiles: ["test"] adminer: profiles: ["tools"] ``` A service with a non-empty `profiles` list is skipped unless one of its profiles is enabled with `--profile test` or `COMPOSE_PROFILES=test,tools`. Naming a profiled service directly on the command line also activates it. This is the clean way to keep a debug UI, a load generator, or a test-fixture seeder in the same file without inflicting them on everyone's `up`. ## Interpolation vs container environment — the classic confusion These are two different systems that both involve "environment variables": 1. **Interpolation (parse time, on the host).** Any `${VAR}`, `${VAR:-default}` (use default if unset *or empty*), `${VAR-default}` (only if unset), or `${VAR:?error}` in the YAML is substituted by the Compose CLI *before* anything runs. Values come from the **shell environment first**, then the `.env` file in the project directory (or the path given by `--env-file`). Escape a literal dollar as `$$`. 2. **Container environment (runtime, inside the container).** `environment:` (map or list) and `env_file:` inject variables into the process. `env_file` values are **not** visible to `${...}` interpolation, and `environment: - DB_PASSWORD` with no value passes the variable through from the host shell. Precedence when the same key is set several ways, strongest first: `docker compose run -e` → `environment:` in the file → `env_file:` → the image's `ENV`. The host shell only participates when you use pass-through syntax or interpolation. The practical consequence: `.env` is for *shaping the file* (image tags, host ports, project name); real container config belongs in `environment:`/`env_file:`, and real secrets belong in neither — they should come from the CI secret store into the shell, or from Docker secrets. ## CI-specific concerns - **Isolate the project.** Parallel jobs on one runner collide over container names and host ports. Set `-p ci-${BUILD_ID}` or `COMPOSE_PROJECT_NAME`, and prefer unpublished ports or `"0:5432"` with `docker compose port` to read back the ephemeral mapping. - **Deterministic waiting.** `up -d --wait --wait-timeout 120` instead of `sleep`. - **Always clean up.** `docker compose -p … down -v --remove-orphans` in a trap/`always` step, otherwise the runner accumulates volumes. - **Pin images** by tag or digest in the base file so dev and CI test the same bits. ## Verify, don't guess `docker compose -f a.yaml -f b.yaml --profile test config` prints the fully merged, fully interpolated document. Any question of the form "which value actually wins?" is answered by that command in one second, and saying so in an interview signals you have debugged this for real.
- An override file lists a single port for a service that already publishes two in the base file. How many ports are published?Three. `ports` is a sequence, and Compose appends sequences across files rather than replacing them, so the override adds to the inherited list. To actually replace it you use the `!override` tag on the list, and `!reset` to clear it entirely; verify with `docker compose config`.
- Why can a variable defined in `env_file:` not be used as `${VAR}` elsewhere in the compose file?They belong to different phases. `${VAR}` is interpolated by the Compose CLI on the host while parsing the file, sourcing values from the shell environment and the `.env` file only. `env_file:` is an instruction to inject variables into the container at runtime, which happens long after parsing, so its contents are invisible to interpolation.
- Two CI jobs on the same runner start the same stack and one fails with a port allocation error. What do you change?Give each job its own project name (`-p ci-$BUILD_ID` or `COMPOSE_PROJECT_NAME`) so container, network and volume names do not collide, and stop pinning host ports — either drop publishing entirely and let tests run inside the network, or publish with only the container port and read back the ephemeral host port with `docker compose port`.
saying these in an interview costs you the question
- Maintaining a full copy of the compose file per environment instead of layering overrides.
- Expecting a later `-f` file to replace an inherited list such as `ports` or `volumes` (they append unless `!override` is used).
- Believing `.env` values are injected into containers automatically, or that `env_file` values are available for `${...}` interpolation.
- Forgetting that passing `-f` explicitly stops `compose.override.yaml` from being loaded.
- Running parallel CI stacks without a distinct project name and then blaming flaky port allocation.