A configuration server like Spring Cloud Config or a KV store like Consul often resolves settings through a layered hierarchy: a base/default configuration, then per-environment overrides, then sometimes per-instance overrides. Describe how such a layered override hierarchy is typically resolved at startup, and why a single flat config file per environment stops scaling as the number of services grows.

## The merge, layer by layer The mechanism behind layered configuration resolution is a **precedence-ordered merge**, conceptually similar to CSS cascading or object inheritance. - A **base** or 'default' profile defines the full set of configuration keys an application understands, each with a reasonable default value - things like connection-pool size, retry counts, log level. - On top of that sits an **environment-specific layer** (commonly named by profile: `dev`, `staging`, `prod`) that only needs to specify the keys whose value actually differs in that environment - a production database host, a staging feature toggle, a dev-only verbose logging flag. - Some systems add a **third layer** below that: per-instance or per-region overrides, useful when, say, one region needs a different timeout because of network latency, or a canary instance needs a different log level for debugging. At application startup, the instance identifies itself (application name, active profile, and optionally instance ID or region) and the configuration client fetches all matching layers from the store, then merges them key-by-key with a defined precedence rule - typically **most-specific-layer-wins**, so an instance-level override beats an environment-level override, which beats the base default. | System | How the hierarchy is expressed | |---|---| | Spring Cloud Config | implements this via file-naming convention (`application.yml`, `application-prod.yml`, `myservice-prod.yml`) and profile activation | | Consul and etcd | implement it via key-prefix hierarchies (`config/base/`, `config/prod/`, `config/prod/instance-3/`) that a client walks and merges | ## The cost of one flat file per environment This pattern exists because configuration has a **Pareto distribution**: the overwhelming majority of keys are the same across every environment, and only a small, predictable subset genuinely needs to differ. If a team instead maintains one flat file per environment - a full `dev.properties`, a full `staging.properties`, a full `prod.properties`, each independently listing every key - two problems compound as the service count grows. 1. **First, duplication cost:** a change to a shared default (say, bumping a default HTTP client timeout from 5s to 10s across the board) now requires editing N files, one per environment, and remembering to do it consistently; miss one and you have silent environment-specific drift that nobody intended. 2. **Second, review and audit cost:** a diff against a flat file doesn't tell a reviewer whether a changed value is intentional environment-specific tuning or an accidental omission of a shared update, because there's no structural signal distinguishing 'this differs on purpose' from 'this differs by mistake.' Layered config makes the intentional deltas the only thing that's visible in the environment-specific layer, which is both smaller to review and self-documenting - anyone reading `prod.yml` sees exactly what makes production different, nothing else. ## The trade-off The trade-off is added indirection and a subtler mental model. - Debugging 'what value will this instance actually use' now requires understanding the merge order and checking multiple layers rather than reading one file top to bottom. - If the precedence rules themselves are non-obvious (or inconsistently applied across tooling), engineers can misjudge which layer wins. - Some config systems make this worse by allowing partial-key overrides inside structured values (e.g., overriding one field of a nested object), which can produce confusing, hard-to-predict merges if the merge semantics aren't well specified (deep-merge vs shallow-replace). ## Where the hierarchy breaks Failure modes tend to appear at the boundaries of the hierarchy. - **A common one:** someone adds a new required key to the base layer, assuming it'll propagate everywhere, but an existing environment layer already defines an older, conflicting key with a similar name, and the override silently wins - the new default never takes effect in that environment and nobody notices until behavior diverges. - **Another:** an instance-level override meant for one canary machine gets accidentally left in place after the canary is retired, so a 'temporary' setting quietly persists in production indefinitely, invisible unless someone thinks to check the instance-level layer specifically. - **A third:** merge order differs subtly between local development tooling and the actual production config-server implementation, so 'it worked when I tested the override locally' doesn't guarantee the same precedence in the real cluster. ## A concrete real-world instance A concrete real-world instance: **Spring Cloud Config Server** backed by a Git repo commonly structures files as - `application.yml` - base defaults for every app - `application-prod.yml` - shared production overrides for all apps - `myservice-prod.yml` - overrides specific to one service in production Spring Boot resolves and merges all three based on the active profile and `spring.application.name`, giving each service a minimal, auditable delta from the shared baseline rather than a monolithic per-environment file.