skip to content

Design a secrets-management strategy for a Spring Boot service across local, CI, and Kubernetes prod. What are the tradeoffs of env vars vs config trees vs an external secrets manager, and how do you keep secrets from leaking through Spring's own surfaces (actuator, logs)?

level: principalimportance: should knowfreq 28%

answer

  1. no secrets in git/image — only placeholders
  2. local untracked file, CI masked vars, prod Secret→configtree
  3. env=leaky+restart; tree=in-place rotate+needs refresh; Vault=audit/lease/rotate+complex
  4. lock down /actuator/env, keep Sanitizer, don't toString secrets
  5. validate required secrets non-placeholder at startup

basics

~20 s

Commit only non-secret defaults. Locally use an untracked file or env vars; in CI use masked pipeline secrets; in prod mount Kubernetes Secrets as a config tree (optional:configtree:) or pull from Vault. Lock down /actuator/env, rely on Boot's sanitizer, and never log secret values.

solid answer

~50 s

The strategy is layered: **git holds no secrets**, only placeholders. **Local** dev reads an untracked `.env`/`application-local.yml` (git-ignored) or env vars. **CI** injects masked pipeline secrets as env vars. **Prod** mounts Kubernetes `Secret` volumes consumed via `spring.config.import=optional:configtree:/etc/secrets/`, ideally sourced from an external manager (Vault, cloud secrets manager) through an operator (External Secrets Operator) or Spring Cloud Vault. `optional:` keeps one artifact runnable everywhere. Env vars are simplest but leak via `/proc`, child processes and crash dumps and can't hot-reload; config-tree files are safer and rotate in place but need a refresh mechanism to re-read; an external manager adds auditing, dynamic/leased credentials and central rotation at the cost of infra. Protect Spring's own surfaces: secure or disable `/actuator/env`/`/actuator/configprops`, keep the default `Sanitizer` redaction, never `toString()` a secret, and prefer binding to types that don't end up in logs.

code

java · 18 lines
java
import jakarta.validation.constraints.NotBlank;
import org.springframework.boot.context.properties.ConfigurationProperties;
import org.springframework.validation.annotation.Validated;

// Fail fast if the prod secret never got injected (placeholder/blank),
// and never leak it via logs/toString.
@Validated
@ConfigurationProperties("app.db")
public final class DbSecrets {
    @NotBlank private final String password;

    public DbSecrets(String password) { this.password = password; }
    public String password() { return password; }

    @Override public String toString() { return "DbSecrets{password=***}"; } // masked
}
// application.yml: spring.config.import=optional:configtree:/etc/secrets/
// management.endpoint.env.show-values=never  (keep Sanitizer redaction)

go deeper

for a junior

Understand secrets don't go in git and are injected at runtime; know actuator can leak config.

for a middle

Describe the local/CI/prod split and the env-var vs mounted-file distinction.

for a senior

Compare all three mechanisms with rotation/leakage tradeoffs and lock down actuator + logging.

for a principal

Own the end-to-end contract: external manager → operator → Secret → config tree, dynamic vs static creds, rotation propagation, fail-fast validation, and defense against Spring's own disclosure surfaces.

## Guiding principle Treat secrets as **runtime inputs, never build-time artifacts**. The repository and the container image contain only **non-sensitive defaults/placeholders**; every real secret is supplied by the environment the app runs in. Spring Boot's precedence (OS env and injected config-data outrank committed files) makes this safe: the injected value always wins over the placeholder. ## Per-environment design - **Local dev**: an **untracked** `application-local.yml` or a `.env` (both git-ignored), activated with `spring.profiles.active=local`, or plain env vars. Import with `optional:` so nothing is required. Never commit real values, even to a `-local` file. - **CI**: use the pipeline's **masked secret store** (GitHub Actions secrets, GitLab CI variables) injected as env vars for integration tests. Masking prevents echoing into logs. Scope to the minimum jobs. - **Kubernetes prod**: mount a `Secret` as a volume → `spring.config.import=optional:configtree:/etc/secrets/`. Prefer sourcing the Secret from an **external manager** rather than hand-managed manifests. ## The three delivery mechanisms — tradeoffs 1. **Environment variables** (`SPRING_*` relaxed binding) - Pros: universal, zero extra infra, trivial in CI and PaaS. - Cons: visible in `/proc/<pid>/environ`, inherited by child processes, frequently captured in crash/heap dumps and misconfigured log lines; **cannot be rotated without a restart**; awkward for large/binary secrets. 2. **Config trees (mounted secret files)** - Pros: value never in the process environment; **files can be rotated in place** by the kubelet; supports binary secrets via `byte[]`; carries `Origin` for debuggability; `optional:` gives one-artifact portability. - Cons: needs a **refresh mechanism** to re-read after rotation (Spring Cloud Kubernetes reload, actuator `/refresh` + `@RefreshScope`, or a watcher); `subPath` mounts don't get updates; filename must match the property key. 3. **External secrets manager (Vault, AWS/GCP Secrets Manager)** - Pros: central **audit log**, **dynamic/leased** credentials (short-lived DB creds), automated **rotation**, fine-grained policy; can integrate as env/tree via External Secrets Operator or natively via **Spring Cloud Vault** (`spring.config.import=vault://…`). - Cons: operational complexity, an availability dependency, auth bootstrapping (the classic "secret-zero" problem), and cost. A common mature setup: **external manager → operator syncs into a k8s Secret → mounted as a config tree** — combining central control with the simple, code-free config-tree consumption. ## Preventing leakage through Spring's own surfaces - **Actuator `/env` and `/configprops`** can expose configuration. Keep them off the public network, require auth, or disable them. Boot ships a **`Sanitizer`** that redacts values for keys matching sensitive patterns (`password`, `secret`, `key`, `token`, credentials, etc.), and `management.endpoint.env.show-values` / `show-details` control disclosure — leave them at safe defaults (`never`/`when-authorized`). - **Logging**: never log a bound secret; avoid putting secrets in exception messages or DTO `toString()`. For records/classes holding secrets, override `toString()` to mask, or wrap in a type that doesn't render the value. - **Heap/thread dumps**: `/actuator/heapdump` and `/actuator/threaddump` can contain secret strings in memory — restrict them. - **Origin/error messages**: config-data errors reference *locations/filenames*, not values, which is safe — but a failed bind that echoes input could leak; validate early with `@Validated` so bad values fail fast without printing. ## Rotation strategy Decide up front how rotation propagates: restart-based (simplest, ties rotation to a rollout), file-watch + `/refresh` for zero-downtime tree rotation, or dynamic leases from Vault where the app renews short-lived credentials. Document which secrets are rotatable live vs. restart-only. ## Failure modes to reason about - Missing mount in prod but `optional:` set → app starts with a **placeholder** and fails obscurely later; consider validating required secrets are non-placeholder at startup. - Env-var name typo → silent fallback to committed default; catch with `@Validated @ConfigurationProperties` and `@NotBlank`. - Secret in a ConfigMap by mistake → plaintext in etcd; enforce with policy/OPA. - Over-broad RBAC on the Secret → mount `readOnly`, least-privilege service accounts.

  • Your Secret is mounted as a config tree and rotated for zero-downtime. How does the running app pick up the new value?
    The kubelet swaps the mounted file in place, but Spring's Environment is read at startup. You need a refresh path — Spring Cloud Kubernetes reload, actuator /refresh with @RefreshScope beans, or a file watcher — to re-read and rebind without restarting.
  • How do you stop a secret from appearing in /actuator/env?
    Keep the endpoint off public networks or behind auth, leave management.endpoint.env.show-values at never/when-authorized, and rely on Boot's Sanitizer which redacts keys matching password/secret/key/token patterns; ideally disable the endpoint in prod if not needed.
  • What's the advantage of Vault dynamic credentials over a static mounted secret?
    Vault can issue short-lived, per-instance leased credentials (e.g. ephemeral DB users) that auto-expire and rotate, shrinking the blast radius of a leak and giving a central audit trail — versus a long-lived static secret shared across instances.

saying these in an interview costs you the question

  • Committing a real secret to a -local or -dev config file 'because it's not prod'
  • Exposing /actuator/env publicly and assuming values are safe
  • Claiming config-tree mounts hot-reload into the Spring Environment with no refresh mechanism
  • Treating env-var injection as equivalent in safety to mounted files or a secrets manager
  • Logging or toString-ing @ConfigurationProperties that hold secrets

context

resolving links…