Why is running the Kubernetes Vertical Pod Autoscaler in Auto mode alongside a Horizontal Pod Autoscaler that targets CPU utilization considered unsafe, and what can you do instead?

## The mechanism, precisely A Horizontal Pod Autoscaler with a `Resource` metric of type `Utilization` does not read raw CPU. For each Pod it computes `usage / request` for the containers it tracks, averages that across ready Pods, and applies the ratio formula: `desiredReplicas = ceil(currentReplicas × currentValue / targetValue)`. The request in that denominator is a **declared** value, not a measured one. VPA's entire job is to change declared requests. So the HPA's input signal moves whenever VPA acts, even if the workload is perfectly steady. ## Walking the loop 1. Steady traffic. Each Pod uses 300m CPU, requests 1000m. Utilization 30%, HPA target 70% — no action. 2. VPA sees consistent 300m usage against a 1000m request and recommends roughly 400m. 3. In Auto mode the updater evicts the Pods; the webhook recreates them with `requests.cpu: 400m`. 4. Same 300m usage, new denominator: utilization is now 75%. Over target, so the HPA scales out. 5. The extra replicas share the same total traffic, so per-Pod usage falls to, say, 200m. 6. VPA now sees 200m and recommends smaller still. Utilization climbs relative to the new request, HPA adds more replicas. 7. Eventually traffic drops, per-Pod usage collapses, the HPA scales in hard, the surviving Pods spike over their now-tiny requests, get throttled, and VPA reacts again. Second-order damage matters as much as the oscillation. Every VPA action in Auto mode is an **eviction**, so this loop produces continuous rolling restarts. Restarts churn connections, cold caches and JVM warm-up, which itself perturbs CPU usage and feeds the loop further. And a request driven very low makes the Pod a poor citizen: cgroup CPU shares are proportional to the request, so it loses arbitration under contention exactly when it needs CPU. ## Official position The VPA documentation states directly that VPA should not be used with an HPA on CPU or memory. It is explicitly compatible with an HPA driven by **custom or external metrics**, because those never reference the resource request. ## The three practical resolutions **1. VPA in `Off` mode.** The recommender still publishes `status.recommendation`; nobody mutates anything. Engineers, or a periodic job that opens a pull request, update the manifests. The HPA keeps a stable denominator. This is the lowest-risk arrangement and is what most mature platforms actually run. **2. Split the resources.** Set `resourcePolicy.containerPolicies[].controlledResources: ["memory"]` so VPA owns memory only, and let the HPA own CPU. This is the widely used split and it is sound, because memory requests are not an HPA input when the HPA targets CPU. Two cautions: memory changes still require a Pod restart in the classic implementation, and if the HPA also has a memory-utilization metric the same conflict returns for memory. Some managed platforms package this split as a supported product feature. **3. Move the HPA off resource utilization.** Target requests per second, queue depth, or another custom or external metric. This is the cleanest fix because horizontal scaling then follows demand directly rather than a proxy that happens to be defined in terms of the request. It costs a metrics pipeline — an adapter serving `custom.metrics.k8s.io` or `external.metrics.k8s.io`. A fourth, weaker option is to use `type: AverageValue` instead of `Utilization` on the CPU metric, which compares absolute millicores against a fixed number and therefore does not divide by the request. It removes the direct coupling, though you then have to keep that absolute target in step with the Pod size VPA is choosing, which reintroduces the coupling in human form. ## Diagnosing it in the wild Symptoms: replica count sawtooths with no matching change in traffic; `kubectl describe hpa` shows repeated SuccessfulRescale events in both directions; Pod ages are uniformly short; the VPA object shows recommendations moving each cycle. Correlate the HPA's `currentCPUUtilizationPercentage` against actual request rate — if utilization moves while request rate does not, something is moving the denominator. ## How to answer Do not just say "they conflict". Say **why**: utilization is a ratio whose denominator VPA owns. Then give the split — VPA on memory, HPA on CPU — or custom metrics, and mention that `Off` mode plus human right-sizing is a perfectly respectable production answer.