Why does helm upgrade leave a chart's crds/ CustomResourceDefinition untouched?

### The mechanism Helm's upgrade logic is built entirely on the release record. Each revision stores the manifest that was rendered from `templates/` in the Secret `sh.helm.release.v1.<name>.v<rev>`; `helm upgrade` renders the new chart, compares it with that stored manifest, and writes the difference. CRDs installed from `crds/` are never rendered and never stored there, so an upgrade has no old copy to compare against and no instruction to apply the new one. It does not skip them because of a bug or a missing flag — there is simply no code path from `helm upgrade` to `crds/`. So `--force` (in Helm 4 a deprecated alias of `--force-replace`) will not do it, `--force-conflicts` will not do it — that flag is about overriding field-manager conflicts under server-side apply — and neither will bumping `version` in `Chart.yaml`. ### Why it is designed this way A CustomResourceDefinition is not a namespaced object belonging to your release. It is cluster-scoped, it is typically shared by every release of that chart in every namespace, and objects that other teams created are validated and stored against it. Two failure modes drove the decision: - **An automated edit can invalidate stored data.** Changing a schema is not a text edit; existing stored objects were written against the old one. Helm has no per-chart knowledge of whether a change is additive or breaking, so it cannot decide when a patch is safe. - **A delete is catastrophic and irreversible.** Removing a CustomResourceDefinition removes every custom resource of that kind across the whole cluster. If Helm managed CRDs as ordinary release resources, dropping the file from a chart, or uninstalling one release out of several, would quietly destroy other people's objects. Given those, refusing to act is the conservative and defensible choice. It is a known sharp edge rather than an oversight, and it has survived every major version so far. ### What this means for a rollback Worth saying out loud in an interview because it surprises people: `helm rollback` re-applies a stored revision's rendered manifest. CRDs were never in that manifest, so rolling a release back from chart 2.6.1 to 2.4.7 leaves whatever CustomResourceDefinition is on the cluster in place. If the newer chart's schema change was additive, that is harmless. If it was not — a field removed, a version un-served — the rolled-back release renders custom resources the current definition rejects, and the rollback itself fails. ### The two honest options **Apply the CRD out of band.** Treat the definition as a separate, deliberate step: read it out of the exact chart version you are about to install with `helm show crds`, apply it, then run the chart upgrade. This keeps `crds/` doing what it is good at — bootstrapping a fresh cluster — while a pipeline step handles the change on clusters that already have the release. **Put the CustomResourceDefinition in templates/ instead.** Then it is an ordinary release resource: templated, gateable on a value, and patched on every upgrade like anything else. What you take on: - **Deletion.** `helm uninstall` now removes it, and every custom resource of that kind with it. `helm.sh/resource-policy: keep` on the manifest prevents that, at the cost of leaving an orphan behind that nothing will ever clean up. - **Ownership.** Only one release can own a cluster-scoped object. Install the same chart in a second namespace and the second install fails, because the object already exists carrying another release's `meta.helm.sh/release-name` and `meta.helm.sh/release-namespace` annotations. - **Ordering.** Helm sorts a CustomResourceDefinition early in its install order, but a custom resource of that new kind rendered by the same release can still be rejected because the API server has not finished registering the kind. `crds/` exists precisely to remove that race. ### The history, briefly The `crds/` directory arrived with chart `apiVersion: v2` as the replacement for the old `crd-install` hook, and the install-once rule has not moved since. If someone asks whether Helm 4 fixed CRD upgrades, the answer is no: Helm 4 changed the default apply strategy, the wait strategy and several flag names, and left `crds/` semantics exactly where they were. Anyone claiming otherwise is guessing.