skip to content

Would you let Helm charts render Secret objects from values across a 12-namespace fleet?

level: principalimportance: should knowfreq 38%

answer

  1. Ask what the chart is allowed to see
  2. One decision: does it reach the values
  3. Count who can read a namespace
  4. Self-contained install has a price
  5. Migration must rotate, not just move

basics

~20 s

For long-lived shared credentials, no: charts should accept the name of a Secret something else owns, so nothing sensitive reaches the values or the release record. Keep the self-contained form only for disposable environments, and rotate before migrating.

solid answer

~50 s

The real question is whether credential material may pass through `.Values` at all. Passing it keeps the install self-contained, but the plaintext then lives in the values file, in the invocation, and twice in every retained release record — so anyone who can read Secrets in a namespace can recover every credential ever passed to any release there. If instead the chart takes `existingSecret` plus a key name, the release record holds only a name, rotation stops being a Helm operation, and the chart can never clobber the value. The costs are honest ones: a bootstrap prerequisite in every environment, no automatic pod roll on rotation because the chart cannot checksum what it never sees, and a rendered manifest that no longer fully describes the system. I would mandate the reference form for shared, long-lived material, allow the self-contained form only in disposable namespaces, and rotate before migrating — old revisions keep the old value.

code

yaml · 10 lines
yaml
{{- if .Values.signing.existingSecret }}
{{- else }}
apiVersion: v1
kind: Secret
metadata:
  name: {{ include "pdfsign.fullname" . }}-signing
type: Opaque
stringData:
  passphrase: {{ required "signing.passphrase or signing.existingSecret is required" .Values.signing.passphrase | quote }}
{{- end }}

go deeper

for a junior

Understand the two shapes a chart can take: it renders the Secret from a value you pass, or it takes the name of a Secret someone else created. Know that the second keeps the credential out of Helm entirely.

for a middle

Explain what each shape stores and where. Be able to describe the existingSecret values contract and how one pod spec consumes either source through a single reference with a default.

for a senior

Argue the tradeoff with the operational costs named: the bootstrap prerequisite, the lost automatic roll on rotation, and the weaker reproducibility. Be able to plan a migration that rotates before it moves.

for a principal

Own the fleet-wide rule and its enforcement point, decide where the self-contained form is still acceptable, and be candid about third-party charts that offer no alternative and about what the estate's existing release records already contain.

### The decision, stated precisely Every chart that handles a credential sits on one side of a line: either the material passes through `.Values` on its way into a rendered `Secret`, or the chart only ever names a Secret that something else created. This is a *values-contract* decision, and across a fleet — say 12 namespaces holding a few dozen releases — it is the single choice that determines how many places plaintext credentials exist. ### What "the chart renders it" really commits you to Take the self-contained form first, because it is the default that upstream charts ship and the one teams reach for: ```yaml signing: passphrase: "" # required ``` The appeal is real. One command installs a working PDF-signing service. `helm template` reproduces the entire release. `helm get values` reconstructs the install. The chart can put a checksum annotation over the rendered credential so pods roll automatically when it changes, which is a genuine operational nicety. The cost is the trail. The value exists in whatever values file or `--set` supplied it, in the job that ran the command, and — twice — in the release record Secret Helm writes for the revision, which holds both the supplied values and the rendered manifest. Retention makes that durable: Helm keeps the last `--history-max` revisions, default 10, so a rotation leaves the old credential sitting in the namespace for another ten upgrades. Multiply by 12 namespaces and every release in them, and the honest description of the estate is: **anyone who can read Secrets in a namespace can recover every credential ever passed to any release in it**. That is usually a bigger grant than the one people think they are making when they hand a team namespace access, and it is what makes this a leadership decision rather than a chart-authoring preference. ### What the reference form costs ```yaml signing: existingSecret: pdfsign-signing passphraseKey: passphrase ``` Now the release record holds a name. `helm get values` is safe to paste into a ticket. Rotation stops being a Helm operation entirely — whoever owns the Secret updates it, with no upgrade and no new revision. And the chart cannot clobber the credential, which retires the whole regeneration failure mode. You pay for it in three places, and a principal answer names them rather than pretending the choice is free: * **The chart no longer installs itself.** There is a prerequisite step, which has to exist in every environment including ephemeral test namespaces and disaster recovery. Teams that skip designing that step end up with a chart that fails on first install in a way that reads like a chart bug. * **You lose the automatic roll on rotation.** The chart cannot checksum content it never sees, so when the Secret changes, pods keep the old value until something restarts them. You need an explicit answer for that — the workload re-reading its credential, or a deliberate restart in the rotation procedure. * **Reproducibility weakens.** The rendered manifest no longer fully describes the running system, because part of the input lives outside the release. That is exactly the property that made it safe, but it does mean a rendered diff is no longer the whole story. ### Where I would land, and how I would say it For a fleet, mandate the reference form for anything long-lived and shared — signing material, database credentials, anything whose compromise is not contained by the namespace. Allow the generate-or-pass form only where the credential's blast radius is the release itself and the environment is disposable, which usually means development and per-branch test namespaces. Write the rule as a values-contract convention every internal chart follows: an `existingSecret` field that, when set, wins, with the self-contained path kept only for the disposable case. Two implementation points make the mandate stick. First, enforcement belongs where charts are reviewed and rendered, not in a document — a render-time check that no rendered `Secret` in a production path carries a value sourced from `.Values`. Second, migration is the hard half: the credentials are already in existing release records, so switching the chart does not clean up history. The migration is *rotate, then move* — the old material stays recoverable in old revisions until it ages out, so it has to stop being valid, not merely stop being passed. Third-party charts complicate this and it is worth conceding: many upstream charts accept only a value, and the practical answer is to use whichever `existingSecret`-style field the chart already offers, and to treat a chart with no such field as a fact to weigh when adopting it — not something to fork over. The trap to avoid is declaring victory on the wrong boundary. Encoding the value, moving it from `--set` to a values file, or reading it with `--set-file` changes who can see it in a terminal and nothing about what Helm stores. The only change that empties the release record is the credential never entering `.Values` in the first place.

  • You switch a chart to existingSecret. Are the old credentials gone?
    No, and that is the part teams miss. The material is still in the release records for the retained revisions — up to `--history-max`, default 10 — and stays recoverable by anyone who can read Secrets in the namespace until further upgrades age those revisions out. The migration order has to be rotate first, then change the chart, so the value sitting in history is no longer valid rather than merely no longer passed.
  • What do you lose operationally by referencing a Secret the chart does not render?
    The automatic roll on rotation. A chart that renders the credential can checksum it and restart pods when it changes; a chart that only names it cannot see the content and so cannot detect a change. You need an explicit substitute — the workload re-reading its credential, or a restart built into the rotation procedure — otherwise pods run on a stale value indefinitely.
  • Upstream charts often accept only a plain value. What do you do about those?
    Use whichever existing-secret-style field the chart already exposes, since most mature ones have one, and treat its absence as a real factor when choosing a chart rather than a reason to fork. Where you must pass a value, contain the consequence: a dedicated namespace, tight read access on Secrets there, and a credential whose scope is limited to that release.
  • How would you enforce the rule rather than document it?
    Check it where charts are rendered. A render of the production values should contain no Secret whose content came from values, and that is mechanically checkable in the pipeline that already builds the manifests. A written convention with no check drifts within a quarter, especially as teams adopt third-party charts that default the other way.

saying these in an interview costs you the question

  • Treating encoding or a values file as the mitigation
  • Assuming existingSecret is free of operational cost
  • Forgetting old revisions still hold the previous credential
  • Mandating a rule with no bootstrap step for fresh namespaces
  • Ignoring that pods will not roll when a referenced Secret rotates
  • Claiming namespace access already limits who sees release values

context