skip to content

A Helm chart contains a CRD and a ClusterRole, but the pipeline identity is namespace-scoped — how do you ship it?

level: seniorimportance: should knowfreq 48%

answer

  1. Render it before you run it
  2. Sort the manifests by scope, not by chart
  3. crds/ is a one-time, privileged write
  4. Cluster-scoped names collide across namespaces

basics

~20 s

Split the install by scope. Have a privileged, rarely-used path install the cluster-scoped parts once — the CRDs from crds/, then the cluster-scoped objects — and let the pipeline install the rest with --skip-crds and the chart's cluster-scoped resources switched off in values.

solid answer

~50 s

First confirm the problem offline: `helm template` the chart and list the kinds, since cluster-scoped objects render with no namespace. Then separate the install by scope rather than widening the pipeline. CRDs under `crds/` are installed only on first install and never upgraded, so an admin can apply them once and the pipeline can pass `--skip-crds` forever after. Cluster-scoped objects in `templates/` usually sit behind a values toggle; turn them off and install them from a separate, privileged chart or run that installs them once. If neither is possible, the honest answer is that this chart needs a cluster-scoped installer and the pipeline should not be it — install it through a different path. Expect a partial release if you simply try: Helm applies objects one at a time with no permission pre-flight, so the namespaced objects are created and the release lands in `failed`.

code

bash · 9 lines
bash
# 1. Find the cluster-scoped kinds before touching a cluster
helm template tiles ./tile-server | grep -E '^kind:' | sort -u

# 2. Privileged, one-time: install the CRDs and the cluster-scoped half
helm install tiles-cluster ./tile-server-cluster

# 3. Per-namespace pipeline: skip the CRDs, disable the cluster-scoped templates
helm upgrade --install tiles ./tile-server -n geo-staging \
  --skip-crds --set rbac.createClusterRole=false

go deeper

for a junior

Recognise that some objects in a chart are cluster-scoped and that -n does not confine them. Knowing you can see this by rendering the chart with helm template is enough at this level.

for a middle

Explain what the install actually does when a write is refused — partial resources, a failed release record — and what crds/ semantics mean: installed once, never upgraded, never deleted, skippable with --skip-crds.

for a senior

Show a plan rather than a flag: classify the chart's kinds by scope, decide which half a privileged path installs once, and account for upgrades, CRD schema changes and teardown of the leftovers.

for a principal

Own the policy question of which charts are allowed to contain cluster-scoped objects at all, how a rare privileged install path is approved and audited, and what you tell chart authors so their charts stay installable by a namespace-scoped identity.

## What actually happens if you just run it Helm renders the whole chart, sorts the manifests into its install order, and writes them one at a time. There is no authorization pre-flight, so the run gets as far as the first object the API server refuses and stops there with a forbidden error naming that resource and the cluster scope. Everything ahead of it exists; everything after it does not; the release is recorded as `failed`, and its stored manifest describes objects some of which are not there. On install, `--rollback-on-failure` (for which `--atomic` is now a deprecated alias) makes Helm clean the failed install up instead of leaving that partial state — but it does not make the install succeed, and Helm 4 defaults `--wait` to `watcher` when you use it. So the first move is not to run it. Render it: ``` helm template tiles ./tile-server -n geo-staging | grep -E '^kind:' | sort | uniq -c ``` Cluster-scoped kinds are the ones that will bite: CustomResourceDefinition, ClusterRole, ClusterRoleBinding, IngressClass, StorageClass, ValidatingWebhookConfiguration, PriorityClass, Namespace. They render with no `namespace:` field, and no amount of `-n` changes that. ## The three real options **1. Pre-install the CRDs and skip them thereafter.** A chart's `crds/` directory is special: those files are never templated, are installed before anything in `templates/`, are skipped when the CRD already exists, and are never upgraded or deleted by Helm. That one-time nature makes them the easiest thing to move out of the pipeline. An operator with cluster scope installs them once — by running the install once themselves, or by applying the files directly — and the pipeline passes `--skip-crds` from then on. The cost is real and worth saying aloud: because Helm never upgrades CRDs, a chart version that changes a CRD schema needs that same privileged path again, and nothing in a normal upgrade will tell you. **2. Turn the chart's cluster-scoped templates off.** Cluster-scoped objects that live in `templates/` are usually guarded by a values toggle — a chart that renders a ClusterRole typically lets you disable the RBAC block and supply an existing identity by name instead. Set the toggle, and the pipeline's install becomes purely namespaced. Whoever owns the cluster installs the cluster-scoped half once, as a separate small chart or a plain manifest. This is the arrangement most platform teams end up with, and it also makes the privileged half reviewable in isolation. **3. Accept that this chart needs a cluster-scoped installer.** Some charts genuinely are cluster-level software — an operator, a controller with webhooks — and pretending otherwise produces a pipeline with cluster-wide power that happens to be used for one chart. The better answer is that such charts install through a different, deliberately rare path with its own approvals, and the per-team pipeline never touches them. If an in-cluster reconciler such as Argo CD or Flux already owns that chart, the pipeline should not be installing it at all. ## The collision nobody predicts Cluster-scoped objects are not namespaced, so two installs of the same chart in different namespaces contend for one name. Chart helpers usually build names from the release name and truncate them to the 63-character limit — a release called `geo-tile-server-staging-eu-west-frankfurt-primary` and one called `geo-tile-server-staging-eu-west-frankfurt-replica` can truncate to the same 63-character ClusterRole name. The second install fails because the existing object carries the first release's `meta.helm.sh/release-name` and `meta.helm.sh/release-namespace` ownership annotations, and Helm refuses to adopt an object owned by another release. Read that error carefully: it is Helm's release-ownership check, not an RBAC failure, and no amount of extra permission fixes it. Naming the cluster-scoped objects with the namespace included, or hoisting them out of the per-tenant chart entirely, does. ## Uninstall is not symmetric If a privileged path created the cluster-scoped objects and the pipeline owns the rest, `helm uninstall` from the pipeline removes only what it can see and is permitted to delete. CRDs are never deleted by Helm under any identity, and objects annotated `helm.sh/resource-policy: keep` survive deliberately. Plan the teardown when you plan the split, or the cluster slowly fills with cluster-scoped leftovers belonging to releases that no longer exist.

  • The failed install left half the release in place. How do you get back to a clean state?
    Decide first whether the release should exist. If it should not, `helm uninstall` it — the release is recorded even though it failed, so Helm knows what it created and can remove it. If it should, fix the permission or the split and rerun `helm upgrade --install`: Helm reconciles against the stored manifest and creates what is missing. Using `--rollback-on-failure` on the original install would have avoided the partial state entirely.
  • Why is a chart's `crds/` directory the easiest cluster-scoped piece to move out of a pipeline?
    Because Helm treats it as a one-time install: files there are never templated, are installed before `templates/`, are skipped when the CRD already exists, and are never upgraded or deleted. So a privileged actor can install them once and every later pipeline run passes `--skip-crds` and needs no cluster scope. The catch is the same property in reverse — a chart version that changes a CRD needs that privileged path again, and an ordinary upgrade will not warn you.
  • Would granting the pipeline permission on just those two kinds be acceptable?
    Rarely, and you should say why. Being able to create ClusterRoles is close to being able to grant yourself anything, so a namespace-scoped pipeline that can write cluster-scoped identity objects is not meaningfully namespace-scoped any more. Being able to create and change CRDs is comparable at the API level. If the choice is between that and a clean split, take the split; if you must grant it, treat that pipeline as a privileged system, not as one team's deploy job.

saying these in an interview costs you the question

  • Grants the pipeline cluster-admin to make it work
  • Assumes -n makes a ClusterRole namespaced
  • Thinks Helm validates permissions before applying
  • Expects helm upgrade to update CRDs later
  • Blames RBAC for a release-ownership collision
  • Forgets cluster-scoped leftovers at uninstall

context

resolving links…