skip to content

After helm uninstall, why do a StatefulSet's PVCs and the chart's CRDs remain?

level: seniorimportance: must knowfreq 58%

answer

  1. Deletion works from a record, not a sweep
  2. Ask who created the object
  3. One directory is exempt by design
  4. Cluster-scoped deletes cross tenant boundaries
  5. Same claim, two owners, two outcomes

basics

~20 s

Helm only deletes objects listed in the release's stored manifest. Claims created by the StatefulSet controller were never rendered, so Helm has no record of them, and CRDs installed from a chart's crds/ directory are deleted by Helm never, by design.

solid answer

~40 s

Helm's delete set is exactly the stored manifest of the release's current revision — it is a list, not a namespace sweep. PersistentVolumeClaims produced from a StatefulSet's `volumeClaimTemplates` are created by the StatefulSet controller as replicas come up, so they appear in no rendered template and Helm has no record of them to delete; whether they are cleaned up is a Kubernetes-side decision, not Helm's. CRDs installed from the chart's `crds/` directory are a deliberate exception: Helm applies them once and never upgrades or deletes them, so uninstall leaves them cluster-wide. Add anything annotated `helm.sh/resource-policy: keep` and you have the three usual survivors. Run `helm get manifest` before the uninstall to see the real delete set, then check the namespace afterwards rather than trusting the exit code.

code

bash · 8 lines
bash
# Capture the delete set before tearing anything down
helm get manifest re-ranker -n platform > /tmp/delete-set.yaml

helm uninstall re-ranker -n platform

# Verify the namespace instead of trusting the exit code
kubectl get pvc -n platform
kubectl get crd | grep reranker

go deeper

for a junior

Know that uninstall does not clear a namespace: persistent claims and CRDs commonly remain. Check with kubectl after the command rather than assuming a green exit means the namespace is empty.

for a middle

Explain the mechanism rather than listing survivors: Helm deletes the objects in the release's stored manifest, so anything a controller created and anything from crds/ was never a candidate. Contrast a rendered claim with a volumeClaimTemplate claim.

for a senior

Demonstrate the operating habit — capture the delete set before teardown, verify the namespace after, and know that a reinstall will silently rebind to surviving claims. Be ready to explain why the crds/ exception protects other tenants.

for a principal

Own the storage and cost consequence of leftovers across an estate, and the rule that cluster-scoped deletion never belongs in an automated teardown. Decide who reclaims orphaned volumes and on what clock.

### The scenario An internal platform chart owned by the infrastructure team ships a recommendation re-ranker as a StatefulSet with 5 replicas, each declaring a 340Gi claim through `volumeClaimTemplates`, plus a CRD in the chart's `crds/` directory for the tuning resource the re-ranker watches. The team decommissions the service: `helm uninstall re-ranker -n platform` returns success, `helm list` shows nothing. A week later someone notices 1,700Gi still provisioned in that namespace and a CRD still registered cluster-wide. Nothing malfunctioned. Every one of those leftovers is Helm behaving exactly as specified. ### The rule underneath all of it Helm deletes from a list. On uninstall it loads the current revision's stored manifest — the rendered YAML kept in the release record — decodes it into objects, and issues a delete for each one. It does not query the API server for objects labelled with the release, and it does not walk the namespace. Three categories therefore survive, for three different reasons. ### 1. Objects a controller created, not Helm `volumeClaimTemplates` is a template evaluated by the StatefulSet controller, not by Helm's renderer. Helm renders the StatefulSet; the controller then creates one claim per replica as pods are scheduled. Those claims exist only in the cluster, never in any rendered manifest, so Helm has no record of them at uninstall time and issues no delete. Whether they are removed when the StatefulSet goes away is decided by Kubernetes' own claim-retention behaviour for StatefulSets — that is a Kubernetes-side setting on the workload, and Helm has no involvement in it either way. The same reasoning covers every controller-created child: nothing that a controller conjures at runtime is in Helm's delete set. There is a consequence worth knowing in both directions. Install the chart again with the same release name and the same replica count, and the StatefulSet controller finds claims already present under the names it would have created and reuses them. The new release comes up on the old data. That is a genuine recovery trick after an accidental uninstall — and a genuine surprise when someone expected a clean slate and got a week-old index. Contrast this with a PersistentVolumeClaim the chart renders directly under `templates/`. That one *is* in the stored manifest, so uninstall deletes it and the data goes with it, unless the chart author annotated it `helm.sh/resource-policy: keep`. Same word — claim — opposite outcome, decided entirely by who created it. ### 2. CRDs from crds/ CRDs placed in the chart's `crds/` directory are applied once, before the templates, and Helm's contract for them is explicit: they are never templated, never upgraded, and **never deleted**. Uninstall does not touch them, and no flag changes that. Removing one is a manual act. That is not an oversight. A CRD is cluster-scoped, and deleting it removes every custom resource of that kind everywhere in the cluster. If uninstalling one team's release could take out another team's custom resources, uninstall would be an unacceptable operation to run at all. Helm makes the destructive step deliberate and human. Again the contrast matters: a chart that renders its CRD under `templates/` instead has put it in the stored manifest, so uninstall *will* delete it — and take every custom resource of that kind cluster-wide with it. Chart authors who do this normally pair it with `helm.sh/resource-policy: keep` precisely to restore the safe behaviour. ### 3. Objects annotated keep The third survivor class is objects the chart marked `helm.sh/resource-policy: keep`. Helm knows about them, would have deleted them, and deliberately does not. ### Diagnosing it in practice Before the uninstall, print the delete set and treat anything not in it as a survivor by default: ```bash helm get manifest re-ranker -n platform > /tmp/delete-set.yaml ``` After the uninstall, verify rather than assume: list the claims and the custom resource definitions and reconcile them against that file. Do not rely on Helm labels to find the leftovers — controller-created claims carry only what the claim template gave them, which may be no Helm metadata at all, and a CRD from `crds/` was applied without release ownership metadata in the first place. ### The answer an interviewer is listening for Not a list of survivors memorised, but the rule that generates the list: Helm deletes what it recorded, minus what it was told to keep, minus the `crds/` exception. Once a candidate states that, they can derive the leftovers for any chart they have never seen — including the ones this leaf did not enumerate.

  • A chart renders a PersistentVolumeClaim directly in templates/ instead. Does uninstall delete that one?
    Yes. A claim rendered under templates/ is in the release's stored manifest, so it is in the delete set and uninstall removes it — data included. The only thing that stops that is helm.sh/resource-policy: keep on the claim. Whether the underlying volume is then destroyed is a Kubernetes-side reclaim decision, not Helm's.
  • You reinstall the chart with the same release name and replica count. What happens to those leftover claims?
    The StatefulSet controller finds claims already present under the names it would create and reuses them, so the new release starts on the old data. That is useful after an accidental uninstall and dangerous when someone expected a clean environment — which is why a decommission should reclaim storage explicitly rather than leaving it for the next install to inherit.
  • How would you actually remove a CRD left behind after uninstalling the last release that used it?
    Deliberately and with a check first: confirm no custom resources of that kind exist anywhere in the cluster, because deleting the definition removes them all, cluster-wide, including other teams' objects. Then delete it as an explicit, logged step outside the Helm operation — never as an automated tail on a teardown pipeline.
  • Why can you not find the leftover claims by filtering on Helm's release labels?
    Because Helm did not create them. A claim from volumeClaimTemplates carries only the metadata the claim template specified, which often includes no Helm labels at all, and a CRD applied from crds/ never received release ownership metadata. Reconcile against the manifest you captured before the uninstall instead.

saying these in an interview costs you the question

  • Thinks uninstall sweeps the namespace by release label
  • Says Helm deletes every PVC the workload used
  • Expects the crds/ directory to be cleaned up on uninstall
  • Blames a Helm bug for the leftover claims
  • Assumes a reinstall starts on empty storage
  • Automates CRD deletion at the end of a teardown

context

resolving links…