skip to content

GitOps

Delivery expressed as a Git repository that an agent continuously reconciles into a cluster, rather than a pipeline that pushes changes at it. Interviewers ask because the model changes who holds cluster credentials, what 'deployed' means, and how drift is handled.

on this pageshow

questions

20

In GitOps, what is the difference between push-based and pull-based delivery to a Kubernetes cluster, and where does the cluster credential live in each model?

level: juniorimportance: must knowfreq 68%

answer

  1. who opens the connection
  2. direction of travel matters
  3. where the kubeconfig is stored
  4. agent inside the cluster, ServiceAccount
  5. credential never leaves the cluster

basics

~20 s

Push delivery means the CI job holds a kubeconfig and runs kubectl or helm against the cluster. Pull delivery means an agent inside the cluster reads manifests from Git and applies them itself, so no outside system holds cluster credentials.

solid answer

~50 s

Both models start from the same commit; what differs is who opens the connection. In **push** delivery the pipeline is the actor: a CI job authenticates to the cluster's API server — usually with a kubeconfig or a cloud identity stored in the CI secret store — and runs `kubectl apply` or `helm upgrade`. In **pull** delivery an agent installed in the cluster (Argo CD and Flux are the usual choices) reads the desired state from a Git repository or an OCI artifact and applies it to its own API server, using an in-cluster ServiceAccount. So under push the cluster credential lives outside the cluster, in the CI system, once per environment; under pull it never leaves the cluster, and what CI needs instead is write access to a repository and a registry. That credential placement is the main reason teams adopt the pull model.

code

bash · 6 lines
bash
# Push model: CI holds cluster credentials and acts on the cluster
set -euo pipefail
echo "$KUBECONFIG_CONTENT" > "$HOME/.kube/config"
chmod 600 "$HOME/.kube/config"
kubectl set image deployment/web web=registry.example.com/web:1.4.2
kubectl rollout status deployment/web --timeout=120s

go deeper

for a junior

Be able to say plainly which side starts the connection: in push, CI runs kubectl or helm against the cluster; in pull, an agent inside the cluster reads Git and applies it. Name where the kubeconfig lives in each.

for a middle

Explain the mechanics: the agent authenticates with an in-cluster ServiceAccount, CI needs only repository and registry write access, and push requires the API server to be reachable from the runner. Mention that short-lived credentials narrow but do not change the push model.

for a senior

Show you know what moved rather than disappeared — Git merge rights become deploy rights, and the agent itself is a privileged component that needs scoping. Note that the pipeline loses synchronous deploy feedback once it stops calling the cluster.

for a principal

Own the tradeoff at fleet scale: how many environments, whose credential blast radius you are shrinking, who operates and upgrades the agents, and whether the audit story is better in Git history or in pipeline logs.

## Two directions of travel "Delivery" here means getting a change from a repository into a running Kubernetes cluster. Both models begin with the same thing: a commit that describes the desired state — a Deployment manifest, a chart's values, a kustomization. What separates them is which side initiates the connection. In **push-based** delivery the CI/CD pipeline is the actor. A job in the pipeline authenticates to the cluster's API server and issues imperative commands against it: `kubectl apply -f`, `kubectl set image`, `helm upgrade --install`, or a vendor CLI wrapping the same calls. The pipeline decides when the deploy happens, watches it, and reports success or failure in the same job. In **pull-based** delivery a piece of software running inside the cluster is the actor. It watches a Git repository (or an OCI artifact holding the same manifests), notices that the declared state differs from what is running, and applies the difference to its own cluster's API server. The pipeline's last deployment-related act is a commit; nothing in CI ever talks to the cluster. ## Where the credential sits This is the part interviewers are actually testing. Under push, something outside the cluster must be able to authenticate as a highly privileged principal to the Kubernetes API — a kubeconfig with a client certificate or token, or a cloud IAM identity mapped to cluster roles. That credential is stored in the CI system, injected into a job that also executes build scripts and third-party dependencies, and it exists once per environment. A team with dev, staging and production has three sets of cluster credentials concentrated in one system. ```bash # Push model: the CI job authenticates to the cluster itself echo "$KUBECONFIG_CONTENT" > "$HOME/.kube/config" kubectl set image deployment/web web=registry.example.com/web:1.4.2 kubectl rollout status deployment/web --timeout=120s ``` Under pull, the credential the agent uses is an in-cluster ServiceAccount bound to roles in that same cluster. It is created at install time and never leaves the cluster boundary — there is nothing to copy into CI, nothing to rotate across systems, and no single external account that can reach every environment. What CI needs instead is write access to the configuration repository and push access to an image registry. ```bash # Pull model: CI's last deployment act is a commit sed -i 's|tag: .*|tag: "1.4.2"|' env/prod/values.yaml git commit -am "prod: web 1.4.2" && git push # an in-cluster agent notices the change and applies it ``` Using short-lived federated credentials instead of a static kubeconfig narrows the push model's exposure, but it does not change the shape: a system outside the cluster is still able to obtain API access, and it is still reachable by whatever runs in that pipeline. ## Direction of network connections The second structural difference follows from the first. Push requires the API server to be reachable from wherever the runner executes — a hosted runner on the public internet, a runner placed inside the VPC, or a VPN/private-link path. Pull requires only outbound connections from the cluster to the Git host and the registry, both ordinary HTTPS or SSH egress. For clusters with no inbound exposure at all, that difference decides the design. ## What the pull model does not change A weak answer treats pull as "the secure one" and stops. It relocates trust rather than removing it: - **Git becomes the control surface.** Anyone who can merge to the watched branch can change production. Review, branch protection and commit signing now carry the weight the cluster credential used to. - **The agent is privileged.** It typically holds broad apply rights inside its cluster, so it is itself a target and should be scoped to the namespaces or tenants it owns. - **Images still come from a registry.** A poisoned image referenced by a legitimate tag is applied just as faithfully. - **Someone still needs cluster access for humans** — debugging, logs, incident response — so the API server does not become unreachable to everyone. ## Saying it in an interview Name the actor and the credential in one breath: push means CI reaches into the cluster with a kubeconfig it stores; pull means the cluster reaches out to Git with credentials that never leave it. Then say what that buys — a smaller and better-scoped blast radius when the CI system is compromised — and what it costs, which is that your pipeline no longer learns synchronously whether the deploy worked.

  • If the pipeline uses short-lived OIDC-federated credentials rather than a stored kubeconfig, is push delivery then equivalent to pull?
    No. Short-lived credentials shrink the window and remove a long-lived secret at rest, which is a genuine improvement, but the shape is unchanged: a system outside the cluster can still obtain API access, and anything that can run in that pipeline can request it. Pull's claim is structural — no external system is authorised to call the API server at all.
  • In the pull model, who is allowed to deploy to production?
    Whoever can get a commit onto the branch or path the agent watches. Deployment authority moves from "who holds the credential" to "who can merge", so branch protection, required review, and restricting which repository paths the agent trusts become the actual access controls.
  • Does adopting a pull-based agent mean nobody needs kubectl access any more?
    No. Humans still need read access for debugging, log retrieval and incident response, and someone must be able to reach the cluster when the agent itself is broken. What goes away is the automated, always-on write credential held by an external system, not human access.

Push is handing the courier a key to your house so they can put the parcel inside. Pull is the household checking the mailbox on its own schedule — the courier never needs a key.

saying these in an interview costs you the question

  • Calling pull "the secure model" without saying which credential moved
  • Thinking the agent authenticates using CI's kubeconfig
  • Assuming pull-based delivery removes the need for any cluster access
  • Believing the commit or manifests differ between the two models
  • Claiming push is fine because the kubeconfig is "in a secret store"

context

open as a page

In a GitOps setup where an in-cluster agent syncs Kubernetes manifests from Git, how do you promote a version that is already running in staging to production?

level: juniorimportance: must knowfreq 72%

basics

~20 s

Promotion is a Git commit, not a deploy job. You change the production manifests to reference the exact image the staging environment already runs, merge that pull request, and the cluster agent converges production onto it.

open as a page

In GitOps, what is the app-of-apps (root application) composition pattern, and what does it change about how a new component reaches a cluster?

level: middleimportance: must knowfreq 55%

basics

~10 s

App-of-apps is a root GitOps definition whose content is more GitOps definitions: the controller reconciles the root, the root creates the children, and the children deploy the workloads. Adding a component becomes one commit.

open as a page

A team says they practise GitOps because their CI pipeline runs kubectl apply -f manifests/ on every merge to main. Which properties of GitOps does that pipeline already satisfy, and which does it miss?

level: middleimportance: must knowfreq 72%

basics

~20 s

That pipeline satisfies two of the four GitOps properties: the desired state is declarative and versioned in Git. It misses the other two, since nothing pulls that state automatically and nothing reconciles continuously, so drift between merges is never seen or corrected.

open as a page

A CI runner that deploys to all your Kubernetes environments is compromised. What does the attacker gain under push-based delivery, and what does moving to pull-based delivery not protect you from?

level: seniorimportance: must knowfreq 58%

basics

~20 s

Under push, the runner holds cluster credentials for every environment, so the attacker gets immediate admin-level access to all of them. Pull removes that credential, but not a poisoned image or a malicious commit, which the in-cluster agent will apply faithfully.

open as a page

In GitOps, what does it mean to bootstrap a cluster, and why is the GitOps agent's own configuration usually kept in the Git path that the agent itself reconciles?

level: juniorimportance: should knowfreq 42%

basics

~20 s

Bootstrapping is the single imperative step that installs the GitOps agent and points it at a Git path. Everything after that, including the agent's own configuration, is reconciled from Git, so a rebuilt cluster converges back automatically.

open as a page

In a Kubernetes cluster managed by a GitOps agent, an engineer edits a live Deployment by hand instead of changing Git. What does the agent do, and what decides whether that edit survives?

level: juniorimportance: should knowfreq 62%

basics

~20 s

The agent's next reconcile compares live state against the revision in Git, sees the difference and reports the resource as out of sync. Whether the edit is reverted depends on whether automatic correction is enabled; with detection only, it is reported but left alone.

open as a page

A GitOps repository contains an operator's CustomResourceDefinitions plus custom resources that use them, and the first sync into a fresh cluster fails on those custom resources. Why does that happen, and what are the ways to order components in GitOps?

level: middleimportance: should knowfreq 48%

basics

~20 s

The controller applies the whole set in one pass, so a custom resource whose CRD is not yet registered is rejected by the API server. GitOps handles this either by retrying until it converges, or by declaring explicit ordering so the CRDs land first.

open as a page

Your team keeps its existing CI pipeline but adopts an in-cluster GitOps agent for Kubernetes deploys. Which responsibilities move to the agent, and what does the CI pipeline lose?

level: middleimportance: should knowfreq 55%

basics

~20 s

Applying manifests and keeping the cluster converged move to the agent; CI keeps building, testing, scanning, publishing the artifact and recording the desired state. The pipeline loses synchronous deploy feedback — its job finishes before the change is actually running.

open as a page

Many teams keep application source code in one repository and its Kubernetes manifests in a separate "config" repository. Why split them for GitOps delivery, and what does the split cost?

level: middleimportance: should knowfreq 60%

basics

~20 s

Separating manifests from source keeps automated version bumps from re-triggering builds, lets deployment changes be reviewed and permissioned differently from code, and lets one app deploy to many environments. The cost is that a single logical change now spans two repositories and two pull requests.

open as a page

Your fleet's GitOps definitions for 50 clusters are generated from one template plus an inventory list rather than 50 hand-written files. What does that generation buy you, and what failure mode does it introduce?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Generation removes copy-paste drift and makes onboarding a cluster a one-line commit against an inventory that becomes the source of truth. The cost is blast radius: one template edit changes all 50 clusters at once, and the reviewed diff shows one line rather than fifty rendered manifests.

open as a page

GitOps says Git is the single source of truth for desired state. In a live Kubernetes cluster, which parts of a running object does that claim not really cover, and how do you stop the agent reporting them as drift forever?

level: seniorimportance: should knowfreq 38%

basics

~20 s

Git owns only the fields you actually declare. Values written after admission — API-server defaults, injected sidecars, autoscaler-managed replica counts, operator patches — and everything under status have other owners, so you either stop declaring that field or exclude it from the diff.

open as a page

A GitOps agent is configured to delete cluster resources that no longer appear in Git. An engineer renames the manifests directory and the next reconcile deletes a live namespace. Why does deletion behave differently from ordinary drift correction, and how would you bound its blast radius?

level: seniorimportance: should knowfreq 40%

basics

~20 s

Deletion (prune) acts on absence, so anything that makes the rendered source look empty — a rename, a wrong path, a branch checkout — reads as an intentional removal. Bound it by scoping ownership narrowly, protecting stateful objects, and never pruning on a render failure.

open as a page

A team keeps long-lived staging and production branches in their Kubernetes manifest repository and promotes by merging staging into production. What goes wrong with that layout, and what is usually preferred?

level: seniorimportance: should knowfreq 52%

basics

~20 s

Environment branches diverge because each carries its own environment-specific values, so every promotion merge conflicts on the same files and drags unrelated changes along. The common alternative is one branch with a directory per environment, where promotion is an explicit edit to the production directory.

open as a page

Kubernetes workloads need Secrets, but plaintext secrets cannot be committed. Compare committing encrypted secrets to Git (SOPS, Sealed Secrets) with having an in-cluster operator fetch values from an external secret manager.

level: seniorimportance: should knowfreq 50%

basics

~20 s

Encrypting secrets into Git keeps one source of truth and works for bootstrap, but rotation needs a commit and old ciphertext lives in history forever. An operator fetching from an external manager keeps only a reference in Git, so rotation is external — at the cost of a live runtime dependency.

open as a page

You lead platform engineering at a company shipping to Kubernetes clusters, VM fleets and managed serverless. Would you mandate pull-based GitOps delivery everywhere, and how would you decide?

level: principalimportance: should knowfreq 40%

basics

~20 s

No. Pull-based delivery needs a reconciling agent for the target's API, which is mature for Kubernetes and rare elsewhere. Mandate it for cluster workloads and keep push with short-lived, narrowly scoped credentials for VMs and serverless.

open as a page

A GitOps agent reconciles a cluster from Git on a fixed interval of a few minutes. What determines how long after a merge the change is actually running, and why do teams keep the periodic reconcile after wiring a push notification from the repository?

level: middleimportance: nice to knowfreq 35%

basics

~20 s

End-to-end latency is the build and publish time, plus the commit landing in the config repository, plus up to one poll interval before the agent notices, plus apply and rollout. A push notification cuts the detection wait but is lossy, and cluster-side drift raises no repository event at all.

open as a page

A production Kubernetes cluster accepts no inbound connections from outside its network. How does pull-based delivery work there, and what do you give up compared with a cluster your CI system can reach?

level: seniorimportance: nice to knowfreq 36%

basics

~20 s

The in-cluster agent makes outbound connections only — to the Git host and the image registry — so no inbound path to the API server is needed at all. You give up synchronous deploy feedback and inbound webhooks, so syncs fall back to interval polling and status must be pushed outward.

open as a page

You are designing GitOps for a platform with many tenant teams across a fleet of Kubernetes clusters. How do you decide between one shared GitOps control plane for everyone and a separate instance per tenant or per cluster?

level: principalimportance: nice to knowfreq 30%

basics

~20 s

Decide on isolation strength, upgrade coupling, and operational cost. A shared control plane gives one inventory and one thing to run but couples every tenant to one version and one failure domain; per-tenant instances give hard isolation at the price of N upgrades and no single view.

open as a page

You own a GitOps manifest repository serving dev, staging and production for a dozen services. How would you design the promotion path — what is automated, where a human belongs, and how do you stop environments diverging?

level: principalimportance: nice to knowfreq 36%

basics

~20 s

Automate the bump into dev, automate staging behind passing tests, and make production an automated pull request that a human approves. Keep environments from diverging by enforcing that only an allowlisted set of keys may differ between overlays, checked on every pull request.

open as a page