skip to content

Desired-State Reconciliation

7 roadmaps4 questions

Kubernetes is declarative: you write spec, controllers keep driving status toward it, and that is why a deleted pod comes back and a hand-edited replica count snaps back. It is the single most asked conceptual Kubernetes question.

part ofKubernetesoverview, primer and where to startread it →
on this pageshow

explore

questions

4

In a Kubernetes object such as a Deployment, what is the difference between the spec section and the status section, and who writes each one?

level: juniorimportance: must knowfreq 72%

answer

  1. spec = desired, status = observed
  2. controllers write status, humans write spec
  3. generation vs observedGeneration
  4. conditions = machine-readable why
  5. status is a separate subresource

basics

~10 s

spec is the desired state you declare; status is the observed state the system reports. You and your tools write spec, controllers write status. Kubernetes keeps acting until status matches spec.

solid answer

~50 s

Nearly every Kubernetes object has two halves. **spec** is the desired state you declare (3 replicas, this image, this port), written by you via kubectl apply, Helm, or a GitOps tool. **status** is the observed state (replicas ready, conditions, assigned IP), written by the controller that owns the object based on what it actually sees. Hand-editing status is pointless: the controller overwrites it on its next pass. The gap between the two drives everything. A controller reads spec, compares it with the real world, acts to close the gap, and records what it observed in status. That loop never stops, which is why Kubernetes self-heals. Practically: you debug by comparing spec with status (kubectl describe, the Conditions list); status.observedGeneration versus metadata.generation tells you whether the controller has even processed your latest spec; and status is usually a separate subresource, so writing it needs its own RBAC verb and updating spec does not clobber it.

code

bash · 3 lines
bash
kubectl get deploy web -o jsonpath='{.spec.replicas}{"\n"}{.status.readyReplicas}{"\n"}'
kubectl get deploy web -o jsonpath='{.metadata.generation} {.status.observedGeneration}{"\n"}'
kubectl get deploy web -o jsonpath='{range .status.conditions[*]}{.type}={.status} {.reason}{"\n"}{end}'

go deeper

for a junior

Recall the definition crisply: spec = what I want, status = what is, controllers close the gap. Give one concrete Deployment example.

for a middle

Add the mechanics: status subresource, conditions structure, generation versus observedGeneration, and why editing status is futile.

for a senior

Frame it as the debugging method — read spec, read status, explain the delta — and mention RBAC separation via the status subresource.

for a principal

Discuss it as an API design principle: derived state must be recomputable, intent must be the single durable source, and CRD authors should keep no unique data in status.

## Two halves of every object A Kubernetes resource is a document with a stable shape: apiVersion, kind, metadata, then almost always spec and status. **spec is desired state.** A declaration of intent authored by a human or a tool: "I want 3 replicas of image app:1.4 with this memory request." It says nothing about how to get there and nothing about what exists right now. It is durable — it stays exactly as written until someone changes it. **status is observed state.** A report written by the controller responsible for that kind: how many replicas are ready, which conditions hold, what IP was assigned, what generation the controller last acted on. It is derived, not authored. If you edit it by hand the owning controller recomputes it from reality and your edit disappears. ## Why the split exists Separating intent from observation is what makes the system declarative. The user writes only intent; the machinery owns the truth. Any component can look at an object and compute the delta (desired minus observed) without knowing what happened before — no event history, no transcript of past commands. If the delta is zero, do nothing. If not, act. It also gives clean ownership boundaries. RBAC can grant a controller permission to write only the status subresource of a resource, not its spec, so a controller cannot silently rewrite your intent. Conversely, a user's `kubectl apply` of the whole document does not overwrite status, because status is served and written through a separate endpoint (`/status`). ## Reading the pair when debugging The most useful debugging habit in Kubernetes is: read spec, read status, and explain the difference. - `spec.replicas: 3` with `status.readyReplicas: 1` means the controller is trying and something is blocking two pods — check events and pod status. - `metadata.generation: 7` with `status.observedGeneration: 5` means the controller has not yet processed your latest edit; it may be backlogged, crash-looping, or not running at all. - The `status.conditions` array is the standard, structured place controllers explain themselves: each condition has a type (Available, Progressing, Ready), a status of True/False/Unknown, plus reason and message strings. Conditions are the machine-readable version of "why isn't this working". ## Not every field follows the pattern A few objects have no meaningful status (ConfigMap, Secret). Some carry state that is neither pure intent nor pure observation — a Service's `spec.clusterIP` is written by the API server on creation and then immutable, and `metadata.deletionTimestamp` plus finalizers control deletion. Custom resources are expected to follow the convention, and CRDs opt in explicitly with a status subresource so controllers can be granted status-only write access. ## What this implies day to day Don't script against status as if you set it. Don't try to "fix" a bad status by editing it. When something is wrong, change spec (or the world) and let the controller re-observe. And when writing your own controller, treat status as strictly derived output: recomputable at any time from the cluster, never the only copy of important information.

  • What does it mean when metadata.generation is higher than status.observedGeneration?
    generation is bumped by the API server every time spec changes; observedGeneration is written by the controller to record the generation it last reconciled. If observedGeneration lags, the controller has not processed your newest spec yet — it may be backlogged, crash-looping, or not running at all. It is the first thing to check when an edit appears to have no effect.
  • Why can't you just edit status to mark a stuck object as healthy?
    status is derived output, not input. The owning controller recomputes it on the next reconcile from what it actually observes, so any manual edit is overwritten within seconds. Fixing the object means changing spec or fixing the underlying condition it is reporting.

A thermostat: spec is the temperature you dialed in, status is the thermometer reading. You set the dial; the thermostat reports and works to close the gap.

saying these in an interview costs you the question

  • Saying users write status or that editing status changes behavior
  • Claiming Kubernetes stops once desired state is reached, rather than continuously re-checking
  • Confusing conditions with events — conditions are current state on the object, events are transient records
  • Thinking spec is updated by controllers to reflect reality (for example, that the Deployment lowers spec.replicas when pods fail)

context

resolving links…

open as a page

You delete a running Pod with kubectl delete pod and an equivalent Pod appears seconds later. Explain the mechanism that recreated it, and why Kubernetes controllers are described as level-triggered rather than edge-triggered.

level: middleimportance: must knowfreq 58%

basics

~20 s

A controller (the ReplicaSet behind the Deployment) constantly compares desired replica count with the pods it actually sees, and creates one when the count is short. Level-triggered means it acts on current state, not on the delete event, so it recovers even if it missed the event.

open as a page

Compare kubectl create, kubectl replace and kubectl apply for updating a live Kubernetes object, and explain what server-side apply changed about tracking field ownership and conflicts.

level: middleimportance: should knowfreq 48%

basics

~20 s

create fails if the object exists; replace overwrites the whole object, dropping fields you omitted; apply merges your manifest with the live object. Server-side apply moves that merge into the API server and records per-field owners in metadata.managedFields, so a conflicting write is rejected until you force it.

open as a page

In a production cluster a Deployment's replica count flips between two values every few seconds and no human is running kubectl. How do you diagnose and fix that kind of fight over a single field?

level: seniorimportance: should knowfreq 34%

basics

~20 s

Two writers each treat that field as theirs and keep correcting each other's drift. Find them via metadata.managedFields, the audit log and controller logs, then give the field exactly one owner — usually by removing it from the declarative manifest and letting the autoscaler or operator own it.

open as a page