skip to content

A team says they practise GitOps because their CI pipeline runs kubectl apply -f manifests/ on every merge to main. Which properties of GitOps does that pipeline already satisfy, and which does it miss?

level: middleimportance: must knowfreq 72%

answer

  1. four properties, not one command
  2. who is watching between merges?
  3. apply acts on files present, only
  4. edge-triggered pipeline versus endless loop
  5. state as a function of the repo

basics

~20 s

That pipeline satisfies two of the four GitOps properties: the desired state is declarative and versioned in Git. It misses the other two, since nothing pulls that state automatically and nothing reconciles continuously, so drift between merges is never seen or corrected.

solid answer

~50 s

GitOps is usually stated as four properties: desired state is declarative; it is versioned and immutable in Git; agents **pull** it automatically; and those agents **continuously reconcile** live state toward it. Apply-on-merge nails the first two — review, history, rollback by revision — and neither of the last two. It is triggered by an event and then stops, so anything that changes in the cluster between merges is invisible. Someone scales a Deployment by hand and it stays scaled; a half-failed apply is never retried; a manifest deleted from the repo keeps running, because `apply` acts on the files that exist, not on the ones that vanished. A reconciler closes that loop by re-deriving the diff on a timer, forever. CI does not disappear — it still builds, tests and publishes the artifact; it just stops holding the cluster credential and pushing.

go deeper

for a junior

Be able to say what the four GitOps properties are and that a merge-triggered apply covers only the declarative and versioned ones. Naming 'nothing checks the cluster between merges' is enough at this level.

for a middle

Explain the mechanics: the pipeline is event-triggered and exits, while an agent re-derives the diff on every pass. Be ready to give concrete consequences — unretried partial applies, undetected manual edits, and manifests deleted from the repo that keep running.

for a senior

Show you have operated both. Describe what you would actually lose or gain in migrating, where the cluster credential moves, and what new operational surface the agent adds. Say plainly when apply-on-merge is still the right choice for a small estate.

for a principal

Own the position that cluster state should be a function of a repository rather than a side effect of a job run, and what that implies organizationally: build and delivery become separate systems with separate owners, audit shifts from pipeline logs to repository history, and every environment needs a reconciler someone maintains.

## The four properties GitOps claims The common formulation (the OpenGitOps project's wording) is four principles: 1. **Declarative** — the desired state of the system is expressed as data, not as a sequence of commands. 2. **Versioned and immutable** — that state lives in a version-controlled store where every revision is addressable and nothing is edited in place. 3. **Pulled automatically** — software agents fetch the desired state themselves rather than waiting to be handed it. 4. **Continuously reconciled** — those agents observe live state and act to make it match, without end. A pipeline that runs `kubectl apply -f manifests/` after a merge satisfies 1 and 2 comfortably, and 3 and 4 not at all. ## What apply-on-merge genuinely buys you Do not dismiss it. Manifests are reviewed in a pull request, the history says who changed what and when, nobody types edits into production, and rolling back is re-running the pipeline at an earlier revision. That is most of the value people notice first, and for a small estate it is a perfectly defensible stopping point. The interview question is not "is this bad", it is "do you know what it does not give you". ## What it cannot do The pipeline acts on an event — a merge — and then exits. Between merges, nothing is looking at the cluster: - **Out-of-band change persists silently.** An engineer scales a Deployment during an incident, an operator patches an annotation, an experiment leaves a resource behind. The next merge only re-applies the files it happens to touch, which may be weeks away, so the cluster and the repo disagree and nobody knows. - **Nothing retries.** If the apply fails halfway — a rejected admission, a transient API error — you have a partly updated cluster and a red pipeline as the only evidence. Someone must notice and re-run. - **Deletions leak.** `apply` acts on the files present in the directory, never on files that were removed: ```bash git rm manifests/legacy-cron.yaml && git commit -m "drop legacy cron" kubectl apply -f manifests/ # the CronJob is still running in the cluster ``` - **There is no live answer to "what is deployed".** You infer it from the last green pipeline log, not from a comparison against reality. A reconciling agent changes all four: it fetches the revision itself, computes a diff against live state on every pass, reports the result as a status you can query and alarm on, and — if you enable it — corrects the difference or deletes what disappeared from the source. ## Convergence versus idempotence, stated precisely Candidates blur these two. **Idempotence** is a property of a single operation: applying it twice leaves the same result as applying it once. `kubectl apply` of a given manifest is idempotent. **Convergence** is a property of a loop over time: repeatedly running it moves the system closer to the desired state and eventually onto it, whatever state it started from. GitOps needs both, but the second is the one apply-on-merge lacks, and it is what makes "just let it run again" a legitimate recovery strategy. A convergent loop does not resume a failed run from a checkpoint — it re-derives the whole diff from live state and does whatever is still outstanding. That also means partial failure is not a special case needing a cleanup path; it is just a diff that has not closed yet. ## Where CI still belongs The strongest version of the answer names the split. CI compiles, tests, and publishes an immutable artifact, then commits the manifest change (typically an image reference) to the config repo. The agent takes it from there. That split is what removes the standing cluster credential from the CI system, which is a separate argument in its own right; here the point is only that GitOps replaces the delivery step, not the build. ## What the interviewer is filtering for One sentence separates the two mental models. Under apply-on-merge, cluster state is a **side effect of the last pipeline run**. Under GitOps, cluster state is a **function of the repository, evaluated continuously**. Everything else — self-healing, prune, sync status, drift alerts — follows from that difference. ## When apply-on-merge is the right call anyway If the target is not a declarative API with controllers, if you have one small environment, or if a reconciling agent is more operational surface than the team can carry, an event-triggered apply is honest engineering. Just do not call it GitOps in an interview, and know which failure modes you have accepted.

  • If the agent does the deploying, what is left for CI to do?
    Everything up to the artifact: compile, test, scan, and publish an immutable image referenced by digest. Then it commits the new reference into the config repo, usually as a pull request. CI still owns build correctness and quality gates; it simply stops holding a credential that can write to the cluster.
  • Does GitOps only work with Kubernetes?
    In principle no — it needs a target with a declarative API and something that can reconcile against it. In practice the ecosystem is Kubernetes-centric because every object is already declarative and controller-driven, so the agent only has to diff and apply. Reconciling arbitrary cloud APIs is possible but far less well served by off-the-shelf tooling.
  • What does 'versioned and immutable' add beyond 'the YAML is in Git'?
    Every desired state becomes an addressable revision rather than a mutable file someone edits. You roll back by pointing the agent at an earlier revision, not by hand-editing live objects, and any past state can be reproduced exactly. It is what makes 'what should be running' a question with a precise answer.

saying these in an interview costs you the question

  • Says GitOps just means keeping YAML in Git
  • Calls any pipeline that runs kubectl apply GitOps
  • Claims adopting GitOps means dropping CI entirely
  • Assumes apply-on-merge undoes manual cluster edits
  • Thinks removing a manifest from Git removes the resource

context