A chart's selectorLabels helper was changed to include the chart version - what breaks on upgrade?
answer
- Two helpers, one a subset of the other
- One field cannot be changed after creation
- The chart label carries a version
- Fresh installs hide it; upgrades expose it
- Rollback restores, deletion is the only fix
basics
~20 sEvery chart-version bump now changes the workload's label selector, and a selector is immutable after creation. The next upgrade is rejected by the API server, the release fails, and no flag gets past it: the object must be deleted and recreated.
solid answer
~40 s`helm create` deliberately writes two partials. `<chart>.labels` carries everything descriptive — `helm.sh/chart` with the chart version in it, `app.kubernetes.io/version` from `.Chart.AppVersion`, `app.kubernetes.io/managed-by` — and it includes `<chart>.selectorLabels`, which holds only the stable pair `app.kubernetes.io/name` and `app.kubernetes.io/instance`. The split exists because a workload's `spec.selector` cannot be changed after the object is created, while its metadata labels can. Folding version-bearing labels into `selectorLabels` means the rendered selector changes on every chart bump, so the next `helm upgrade` is rejected by the API server on an immutable field. The release goes to `failed`; `helm rollback` restores the previous revision's selector and works, but shipping the new selector requires deleting the workload object and letting Helm recreate it, with the downtime that implies.
code
yaml · 13 lines{{- define "platform-transcoder-euw-workers.labels" -}}
helm.sh/chart: {{ include "platform-transcoder-euw-workers.chart" . }}
{{ include "platform-transcoder-euw-workers.selectorLabels" . }}
{{- if .Chart.AppVersion }}
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }}
{{- end }}
app.kubernetes.io/managed-by: {{ .Release.Service }}
{{- end }}
{{- define "platform-transcoder-euw-workers.selectorLabels" -}}
app.kubernetes.io/name: {{ include "platform-transcoder-euw-workers.name" . }}
app.kubernetes.io/instance: {{ .Release.Name }}
{{- end }}go deeper
Know that a chart from helm create has two label helpers, that the selector one is deliberately smaller, and that the selector is the part you must not edit once releases are live.
Explain what each helper emits and where each is wired — full labels on metadata, the small pair on the workload and Service selectors — and why the version-bearing labels are excluded from the selector.
Walk the incident: a clean install hides the defect, the next chart bump fails the upgrade on an immutable field, rollback restores service, and shipping the change means replacing the object. Say what you would have caught it with in review or CI.
Own the policy: selector labels are a frozen interface across the chart estate, changing one is a migration with a downtime plan, and CI should diff the rendered selector across chart versions so the question never reaches production.
## The two helpers and why there are two A chart from `helm create` contains a pair of label partials that look redundant until you see what each is wired to: ``` {{- define "platform-transcoder-euw-workers.labels" -}} helm.sh/chart: {{ include "platform-transcoder-euw-workers.chart" . }} {{ include "platform-transcoder-euw-workers.selectorLabels" . }} {{- if .Chart.AppVersion }} app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} {{- end }} app.kubernetes.io/managed-by: {{ .Release.Service }} {{- end }} {{- define "platform-transcoder-euw-workers.selectorLabels" -}} app.kubernetes.io/name: {{ include "platform-transcoder-euw-workers.name" . }} app.kubernetes.io/instance: {{ .Release.Name }} {{- end }} ``` The outer helper is a superset of the inner one. `metadata.labels` on every object gets the full set; the workload's `spec.selector.matchLabels` and the Service's `spec.selector` get only the inner pair. Three of the four labels in the full set move: `helm.sh/chart` embeds the chart version, `app.kubernetes.io/version` tracks `appVersion`, and `managed-by` is stable in practice but conceptually descriptive. The inner pair is chosen precisely because nothing routine changes it — the chart name and the release name are fixed for the life of the release. ## What breaks A workload's label selector is immutable once the object exists. That is not a Helm rule and no Helm setting relaxes it. So the moment `selectorLabels` starts emitting `helm.sh/chart`, the chart has coupled an immutable field to a value that changes every time anyone publishes a new chart version. Picture an internal platform chart owned by the infrastructure team, driving a media-transcoding pipeline across 11 releases. Someone "tidies up" the helpers so that the selector matches the full label set. The change ships with chart version 4.2.9 and every release installs cleanly, because a fresh install writes whatever selector it is given. Nothing looks wrong for weeks. Then chart 4.3.0 is published, and the first `helm upgrade` renders a selector containing `helm.sh/chart: platform-transcoder-euw-workers-4.3.0` where the live object has `4.2.9`. The API server rejects the update on the immutable field. Helm marks the release `failed`, and — because Helm 4 waits for workloads only when asked — the error surfaces immediately as an apply error rather than as a timeout. ## Why the usual escapes do not help This is where candidates reach for flags. Server-side apply, the default write path for a Helm 4 install, does not make an immutable field mutable; field ownership decides *who* may write a field, not *whether* it may change. Replacement-style update flags are not a dependable escape either, because the rejection comes from the API server's validation of the object, not from how Helm chose to send it. The only reliable path is to remove the workload object and let Helm recreate it, which means a gap in service unless you plan a parallel cutover. `helm rollback` does work, and it is the right first move during the incident: the previous revision's manifests carry the old selector, so applying them is a legal update and the release returns to `deployed`. That buys time — it does not ship the new chart. ## The related trap: selector and pod labels drifting apart The mirror-image mistake is to make the selector *smaller* than the pod template's labels by accident — for example by pointing `spec.selector.matchLabels` at the full `labels` helper while the pod template uses something else. The selector must be satisfied by the pod template's labels; when it is not, the object is rejected outright at creation. The scaffold avoids both mistakes by including the selector pair inside the full set: pod labels are always a superset of the selector, by construction. ## How to keep it from happening Treat `selectorLabels` as a frozen interface. Anything derived from `.Chart.Version`, `.Chart.AppVersion`, an image tag, a commit SHA or a timestamp belongs in `labels` and never in `selectorLabels`; a comment in `_helpers.tpl` saying so costs one line and saves an outage. In review, any diff touching the `selectorLabels` define deserves the same scrutiny as a database migration. A render-time check in CI helps too: render the chart at two different chart versions and assert the selector block is byte-identical. That catches the change on the pull request rather than on the eleventh release.
- Would running the upgrade with automatic rollback on failure have helped?It would have contained the damage, not shipped the change. Helm 4's `--rollback-on-failure` — the flag Helm 3 called `--atomic` — reverts the release to the previous revision when the upgrade fails, so you end up back on the working selector instead of a half-applied release stuck in `failed`. The immutable field is still immutable; the new selector cannot be delivered by any flag.
- How do you actually ship a chart that must change the selector?Accept that the object is replaced. Either delete the workload and let the next upgrade recreate it during a maintenance window, or stand up the new object under a different name, shift traffic, then remove the old one. Both are deliberate operations with a rollback plan; neither is something to discover mid-upgrade on eleven releases.
- Which labels are safe to add to the full labels helper at any time?Anything descriptive: team, cost centre, chart version, app version, a component name. Metadata labels can change freely on an upgrade, and dashboards and queries built on them keep working. The rule is one-directional — a label may graduate from nowhere into `labels`, but a label already in `selectorLabels` can never be removed or changed for a live release.
The selector is the lock and the pod labels are the key ring: you can keep adding keys, but recutting the lock while the door is closed is not on offer.
saying these in an interview costs you the question
- Saying more labels in the selector is always safer
- Claiming a force or replace flag pushes the new selector through
- Believing server-side apply makes an immutable field writable
- Assuming selector labels must equal the metadata labels
- Thinking a clean install proves the change is upgrade-safe
- Expecting Helm to recreate the workload automatically