A Helm pre-upgrade hook Job fails because a ConfigMap the same chart renders does not exist yet - why?

### Two phases, and the weight only sorts inside one A Helm operation is a sequence of phases, not one big apply. On an upgrade, Helm renders the chart, records a new pending revision, runs the `pre-upgrade` hooks to completion, applies the chart's ordinary manifests, and then runs the `post-upgrade` hooks. `helm.sh/hook-weight` orders the hooks *inside* a single one of those hook phases. It is compared only against the weights of the other hooks firing on the same event. It is not a global position in the release, and there is no weight - however large or however negative - that places a hook between two of the chart's ordinary resources. That is the whole shape of the problem: the ordering the author wants crosses a boundary the annotation cannot cross. ### Why it looks intermittent A chat fan-out service whose chart is packaged by CI and pushed to an OCI registry might carry a `pre-upgrade` Job that rebuilds routing shards and reads the shard count from a ConfigMap the chart also renders. The chart installs fine on a fresh namespace only if the Job tolerates the ConfigMap's absence; more often the first install fails outright. On an upgrade it usually *succeeds*, because the ConfigMap from the previous revision is still in the cluster - the hook reads yesterday's shard count and nobody notices. The failure surfaces the day the upgrade is the one that changes the shard count: the hook runs before the new ConfigMap is applied, reads the old value, and either errors or, far worse, quietly does the wrong work. If the hook blocks waiting for content that will only exist after it finishes, it deadlocks and dies at the command's hook timeout - a 96-second timeout on that Job just means you find out 96 seconds sooner. ### Fix one: bring the data into the hook phase Anything a hook needs can itself be a hook. Annotate a copy of the ConfigMap as a `pre-upgrade` hook with a lower weight than the Job: ```yaml apiVersion: v1 kind: ConfigMap metadata: name: fanout-reshard-input annotations: "helm.sh/hook": pre-upgrade "helm.sh/hook-weight": "-10" data: shards: "48" ``` Now both objects are in the same pool, Helm sorts `-10` before the Job's `0`, and the ConfigMap exists when the Job starts. Two consequences to state before an interviewer states them for you: this object is now a hook, so it lives outside the release's ordinary manifest set and its lifecycle is governed by the hook delete-policy annotation rather than by ordinary upgrade behaviour; and you now have two ConfigMaps rendering the same content, so generate both from one template helper rather than letting them drift. ### Fix two: render the value into the hook itself Often the hook does not need a ConfigMap at all - it needs a value. Passing it through `args` or `env` rendered from `.Values` removes the ordering problem instead of solving it, and it is the cheapest option when the payload is small. ### Fix three: move to a post-* event If the work genuinely belongs after the chart's resources exist, a `post-upgrade` hook is the correct event, and then the ConfigMap has been applied before the hook runs. The trap is readiness rather than existence: Helm always waits for each *hook* to finish, but whether it waits for the chart's own workloads before running the post phase depends on the wait strategy you asked for. In Helm 4, `--wait` is strategy-valued (`watcher`, `legacy`, `hookOnly`) and omitting it means `hookOnly` - hooks are waited for, workloads are not - so a `post-upgrade` hook can start while the new Pods are still rolling. Ask for a waiting strategy if the hook needs the new version actually serving. Helm 3 behaves comparably by default: no `--wait` means no wait on workloads, and its `--wait` is a plain boolean. ### What not to do Do not put a large negative weight on the ConfigMap in the hope of dragging an ordinary manifest earlier - weights on non-hook manifests are ignored. Do not add a sleep to the hook: the ConfigMap is not late, it is in a phase that has not started. And do not reach for a separate pre-apply step in the pipeline that applies the ConfigMap out of band, because you have then split one release into two sources of truth, and a rollback restores only half of it.