skip to content

You flip a bundled database subchart off on a live Helm release. What happens to the objects it already deployed?

level: seniorimportance: should knowfreq 36%

answer

  1. An upgrade removes what vanished
  2. The stored manifest is the reference
  3. Only rendered objects can be deleted
  4. One annotation buys an exemption
  5. Rollback restores objects, not data

basics

~20 s

They are deleted. On upgrade Helm compares the previous revision's stored manifest with the newly rendered one and removes what disappeared, so a disabled subchart takes its objects - including any it rendered to hold data - with it. Only helm.sh/resource-policy: keep spares one.

solid answer

~50 s

Turning a condition off is not a no-op on an existing release. Helm renders the new manifest, diffs it against the manifest stored with the previous revision, and deletes the objects that are no longer present - the disabled subchart's Deployments, Services, Secrets and any storage claim its templates rendered. Objects Helm never rendered are not in that manifest and are therefore never deleted, which is the only reason some bundled databases survive the switch at all. To keep something deliberately, annotate it `helm.sh/resource-policy: keep` while it is still being rendered; Helm then leaves it in the cluster instead of deleting it, at the cost of it becoming untracked. The safe sequence for a real cutover is: render a diff and read exactly which objects vanish, migrate and verify the data against the external service first, then flip the switch in its own revision so the change is trivially reversible.

code

bash · 9 lines
bash
# 1. what exactly disappears if the bundled database is switched off?
helm get manifest billing -n billing > /tmp/before.yaml
helm template billing ./billing -f prod-values.yaml \
  --set billing-db.enabled=false > /tmp/after.yaml
diff /tmp/before.yaml /tmp/after.yaml

# 2. rehearse the upgrade itself without applying it
helm upgrade billing ./billing -f prod-values.yaml \
  --set billing-db.enabled=false --dry-run=server

go deeper

for a junior

Know that switching a bundled component off on an existing release is a destructive change, not a pause - Helm removes what the chart no longer renders.

for a middle

Explain the mechanism: the new render is diffed against the manifest stored with the previous revision, and objects that vanished are deleted unless annotated to be kept.

for a senior

Show the operational sequence - preview the diff, migrate and verify first, flip in an isolated revision, back the data up - and be honest that rollback recreates objects, not state.

for a principal

Own the policy for a fleet: which optional components are allowed to hold production state at all, who reviews a values change that deletes storage, and what guardrails make that diff visible before it merges.

### What an upgrade actually removes Helm stores the rendered manifest of every revision in the release record. When you upgrade, it renders the new manifest and works out three sets: objects that are new, objects that exist in both, and objects present in the old manifest but absent from the new one. That third set is deleted. Nothing about that logic is special-cased for subcharts - a dependency whose condition flipped to false simply stops producing output, so every object it used to produce lands in the delete set. That is the correct behaviour and it is what makes a chart declarative, but it means "switch the bundled database off" and "delete the bundled database" are the same operation. If the subchart rendered a PersistentVolumeClaim, that claim is in the old manifest, and it goes. The complement is equally important: objects Helm never rendered are not in the manifest, so Helm never deletes them. A claim created by a controller on the chart's behalf rather than by a template is invisible to this diff and will be left behind, orphaned. Which of those two applies to your bundled datastore is a question about that specific chart's templates, and the only reliable way to answer it is to look at the manifest the release recorded. ### A worked cutover A billing platform runs twelve service charts that all depend on a shared library chart for their helper templates. One of them, the subscription-billing chart, bundles a database subchart so a developer gets a working stack from a single install. Production is moving to a managed database, so the switch has to be flipped on a release that has been running for months. Done carelessly - one upgrade that changes the external host and sets the condition false at the same time - the upgrade deletes the in-cluster database in the same operation that repoints the application at a host nobody has verified. Done carefully, it is three steps: 1. **See the blast radius.** Render the chart with the new values and diff it against what the release currently has. The output names every object that disappears. Read that list out loud before doing anything. 2. **Migrate and cut over first.** Point the workload at the external database, in its own revision, while the bundled one is still running and still holds the data. Verify the cron and the request path actually work against it. The cutover in our case cost 96 seconds of write pause, planned and announced. 3. **Disable in its own revision.** Only then flip the condition, with nothing else in the diff. A revision that changes exactly one thing is a revision you can reason about, and its predecessor is a coherent state to roll back to. ### Keeping something on purpose `helm.sh/resource-policy: keep` is the annotation that exempts a resource from deletion when a Helm operation would otherwise remove it. It has to be on the resource as the chart renders it - it is a property of the manifest, not something you can bolt on afterwards and expect Helm to notice. Kept resources survive, but they stop being managed: the release no longer tracks them, nothing reconciles them, and a later attempt to have the chart own that name again meets an object already sitting there without the release's ownership metadata. Helm refuses that adoption by default; `--take-ownership` exists to force it, and reaching for it should be a deliberate decision rather than a reflex. ### Rollback is not undelete Rolling back to the previous revision re-renders that revision's stored manifest and re-applies it, so the subchart's objects come back - empty. A rollback restores object definitions, never the contents of storage that was released with them. Treat the data as gone the moment the claim is deleted, and plan the backup accordingly. ### What does not change Disabling a dependency changes only rendering. The entry stays in `Chart.yaml`, the pin stays in `Chart.lock`, the archive stays under `charts/`, and the packaged chart still ships it - so re-enabling it later is a values change, not a repackage. That asymmetry is the feature: one chart, two shapes, and no fork to maintain across a fleet of services.

  • Does rolling back the upgrade bring the bundled database back?
    It brings the objects back, not the data. A rollback re-renders and re-applies the manifest stored with the target revision, so the subchart's resources are recreated - but any storage that was deleted with them is gone, and the recreated database comes up empty. Plan a backup before the flip; do not treat rollback as an undo for state.
  • What is the drawback of keeping the bundled database's storage with resource-policy keep?
    It survives, but the release stops tracking it: nothing reconciles it, it does not appear in the release's manifest, and it will quietly outlive the chart. If the chart later renders an object of the same name, Helm refuses to adopt one it does not own, and forcing that adoption is a deliberate act. Keep is an escape hatch for state, not a default.
  • Why should the condition flip be its own revision rather than part of the migration change?
    So the diff is one line and the previous revision is a coherent state. Bundling the deletion with a repointing change means a single upgrade both removes the old datastore and sends traffic somewhere unverified, and rolling back returns you to a revision whose database no longer holds the writes made since. Separate revisions keep each step reversible on its own.

saying these in an interview costs you the question

  • Believing a disabled subchart's resources are simply left running
  • Expecting rollback to restore deleted storage and its contents
  • Adding resource-policy keep after the resource stopped rendering
  • Flipping the switch in the same upgrade as the data migration
  • Assuming a kept resource is still managed by the release
  • Thinking disabling removes the dependency from the packaged chart

context