skip to content

Why can installing one Helm chart a second time under a new release name still fail on cluster-scoped objects?

level: seniorimportance: should knowfreq 44%

answer

  1. Isolation reaches as far as the objects allow
  2. Which kinds have no namespace
  3. The error names an object outside both namespaces
  4. `crds/` skipped when present, never upgraded
  5. Split the cluster-scoped half into one release

basics

~20 s

Release names are unique per namespace, but ClusterRoles, CustomResourceDefinitions and webhook configurations have one cluster-wide name. A second release of a chart that renders them collides on objects the first release already owns, whatever namespaces the two live in.

solid answer

~40 s

Helm isolates releases by namespace, and that isolation only reaches as far as the objects being namespaced. A chart that also renders cluster-scoped objects — a ClusterRole and its binding, a ValidatingWebhookConfiguration, a PriorityClass, a CustomResourceDefinition under `templates/` — renders the same single name for every release, so the second install fails on an object that already exists and belongs to the first release. CRDs shipped under the chart's `crds/` directory behave differently: Helm installs them only when absent and skips them when present, so a second release does not collide there, but it also silently accepts whatever version the first release installed, since `crds/` definitions are never upgraded. The fix is to split the chart: one release owns the cluster-scoped half, each tenant namespace owns a namespaced-only release, usually gated by a value.

code

yaml · 13 lines
yaml
clusterResources:
  create: true
---
{{- if .Values.clusterResources.create }}
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: {{ .Release.Name }}-{{ .Release.Namespace }}-transcoder
rules:
  - apiGroups: ["media.example.com"]
    resources: ["transcodejobs"]
    verbs: ["get", "list", "watch", "update"]
{{- end }}

go deeper

for a junior

Know that some Kubernetes objects a chart renders do not live in a namespace at all, and that Helm's per-namespace release naming does nothing for those. Being able to name one such kind is enough here.

for a middle

Explain the mechanism: one cluster-wide name per cluster-scoped object, so the second release renders a name the first already owns and the install refuses. Know that crds/ definitions are skipped when present rather than colliding.

for a senior

Demonstrate the operating consequence: the shared CRD that never gets upgraded, the uninstall that pulls shared objects out from under other tenants, and the split into one cluster-scoped release plus one release per tenant. Say how you would test it before promising it.

for a principal

Own the multi-tenancy model: who is allowed to install the cluster-scoped half, what a chart must satisfy before it is declared multi-instance, and whether tenant isolation should rest on namespaces at all when the workload needs cluster-wide definitions.

## The scenario You run a media-transcoding pipeline and you want one operator chart installed once per tenant: six tenant namespaces, six releases, six copies of a 240-line values file with different queue sizes and node selectors. Release names are unique per namespace, so the naming looks fine. The first install succeeds. The second one fails, and the object it names is not in any of the namespaces you were thinking about. ## Why the second install fails Helm scopes a *release* to a namespace. It does not scope the *objects a chart renders* to a namespace — the chart does that, by rendering namespaced kinds. Operator charts almost never render only namespaced kinds. A typical one ships: - a `ClusterRole` and `ClusterRoleBinding` so the controller can watch its custom resources across the cluster; - a `ValidatingWebhookConfiguration` or `MutatingWebhookConfiguration`; - a `PriorityClass` for the controller pod, or a `StorageClass` for the scratch volumes the transcoder uses; - `CustomResourceDefinition` objects, either under `templates/` or under `crds/`. Every one of those kinds is cluster-scoped: there is exactly one `ClusterRole` named `transcoder-operator` in the whole cluster, and it belongs to whichever release created it. The second release renders the same name — because the chart author wrote it as a constant, or because the fullname helper was applied to the namespaced objects but not to this one — and the install refuses: that object already exists and belongs to another release. Helm does not merge, does not rename, and does not take it over by default. The failure is often confusing because the error names a cluster-scoped object while the operator was thinking in namespaces. "But the release names are different and they're in different namespaces" is exactly the misconception the question is testing. ## The `crds/` exception, which is not the relief it looks like CustomResourceDefinitions are the special case. If the chart puts them under `templates/`, they are ordinary rendered objects and behave exactly like the ClusterRole above: the second release collides. If the chart puts them in the top-level `crds/` directory, Helm treats them specially. They are never templated, they are installed before the rest of the release, and — the part that matters here — Helm skips them when a definition of that name already exists. So the second release installs cleanly. What you have bought is not isolation but sharing: both releases now depend on one set of definitions, whose version is whatever the *first* install put there. Helm does not upgrade `crds/` definitions on a later `helm upgrade`, in any version, so bumping the chart in tenant two does not update the schema tenant one installed. If the newer chart's controller expects a field that the older definition does not accept, you get a failure at custom-resource-creation time, far away from the install that caused it. ## What to do instead The general shape of the fix is to stop pretending the cluster-scoped half is per-tenant, because it is not. **Split the chart.** One chart (or one sub-chart, or one value-gated block) owns the cluster-scoped objects and is installed exactly once, by whoever owns the cluster. A second, namespaced-only chart is installed once per tenant. This mirrors how operators are usually shipped in practice: an operator install and a per-namespace instance. **Gate the cluster-scoped block behind a value.** A single chart with something like `clusterResources.create` defaulting to true, set to false for every release after the first, works but is fragile: whoever holds the true has to be upgraded first, and uninstalling that release takes the shared objects out from under the others. **Template the names when the objects genuinely are per-release.** Some cluster-scoped objects can legitimately exist once per tenant — a ClusterRole restricted to that tenant's resources, say. Then the name has to include the release name *and* the namespace, because two releases in different namespaces may share a release name and would otherwise still collide. **Decide about the CRDs explicitly.** Either the cluster owns them (install them once, with the shared release, and upgrade them as a deliberate act) or you accept that they are shared and pin every tenant to a compatible chart version range. What you must not do is assume a per-tenant `helm upgrade` keeps their schema current. ## The check that would have caught it Before promising a chart is multi-instance, install it twice into two scratch namespaces on a throwaway cluster and try upgrading only one of them. Reading the templates finds the obvious hardcoded name; installing it twice finds the webhook configuration nobody remembered was cluster-scoped, and upgrading one release afterwards finds the CRD that never moved.

  • The chart ships its CustomResourceDefinitions in `crds/`, so the second install did not collide. Why is that still a problem?
    Because Helm installs a `crds/` definition only when it is absent and never upgrades it afterwards. The second release silently adopted whatever version the first install created, and bumping the chart for one tenant does not move the schema for anyone. You end up with controllers expecting fields the live definition does not accept. Treat those definitions as cluster-owned and upgrade them deliberately, not as a side effect of a tenant deploy.
  • How would you verify that a chart really is safe to install twice before you promise it to tenants?
    Install it into two scratch namespaces on a throwaway cluster, then upgrade only one of them and uninstall only one of them. The double install exposes cluster-scoped names and untemplated objects, the single upgrade exposes shared definitions that do not move, and the single uninstall exposes shared objects the other release still needs. Reading the templates finds only the obvious cases.
  • Would giving each tenant its own release name but the same namespace change any of this?
    Not for the cluster-scoped objects — they have one name cluster-wide regardless of namespaces or release names. It would make things stricter for the namespaced objects, since two releases in one namespace also need every namespaced name to differ, which the release-name-derived fullname helper handles. The cluster-scoped half still has to be factored out and installed once.

saying these in an interview costs you the question

  • Says different namespaces make two releases fully independent
  • Thinks Helm namespaces a ClusterRole for you
  • Expects a per-tenant upgrade to update `crds/` definitions
  • Claims Helm merges cluster objects shared by two releases
  • Suggests deleting the object by hand as the standard fix
  • Cannot name a single cluster-scoped kind a chart renders

context