After a clinical-records portal's pipeline switched to `kubectl apply --server-side`, deploys fail with "Apply failed with 1 conflict" on the Kubernetes Deployment's `.spec.replicas`. How do you diagnose and resolve it?

seniorshould knowfreq 44%

answer

per-field owners on the server
Apply conflicts, Update takes over
read the manager and subresource
force is a transfer, not a fix
stop declaring what others own

basics

~20 s

Server-side apply records an owning field manager for every field in managedFields. The pipeline's manifest sets replicas to a value another manager, here the autoscaler writing through the scale subresource, owns with a different value, so the server rejects it.

solid answer

~50 s

Under server-side apply the API server does the merge and records, in metadata.managedFields, which field manager owns each field. The pipeline's manager is kubectl by default. If an apply tries to set a field to a value different from what another manager owns, the server returns 409 Conflict and names that manager. Diagnose it with kubectl get deploy records-portal -o yaml --show-managed-fields. The message and entry show an Update from the autoscaler with subresource: scale. That tells you the pipeline and the autoscaler both claim replicas. --force-conflicts would take ownership and set 7 again, and the autoscaler would then fight it. The real fix is to stop declaring replicas. Once the autoscaler's Update has taken ownership, removing the field from the manifest is safe. If kubectl were still the only owner, removing it would drop the Deployment to the default of 1.

Under server-side apply the API server does the merge and records, in `metadata.managedFields`, which **field manager** owns each field. The pipeline's manager is `kubectl` by default. If an apply tries to set a field to a value different from what another manager owns, the server returns **409 Conflict** and names that manager. Diagnose it with `kubectl get deploy records-portal -o yaml --show-managed-fields`. The message and entry show an `Update` from the autoscaler with `subresource: scale`. That tells you the pipeline and the autoscaler both claim `replicas`. `--force-conflicts` would take ownership and set 7 again, and the autoscaler would then fight it. The real fix is to stop declaring `replicas`. Once the autoscaler's Update has taken ownership, removing the field from the manifest is safe. If `kubectl` were still the only owner, removing it would drop the Deployment to the default of 1.

code

bash · 3 lines

bash

kubectl get deployment records-portal -o yaml --show-managed-fields
kubectl diff --server-side -f records-portal.yaml
kubectl apply --server-side --field-manager=records-pipeline -f records-portal.yaml

go deeper

for a junior

Know that server-side apply remembers which tool set each field, and that a conflict means another tool owns a field your file tries to change.

for a middle

Explain the managedFields entry fields, the difference between Apply and Update operations, and what co-ownership and --force-conflicts each do.

for a senior

Show the diagnosis from the conflict message and --show-managed-fields, refuse to force against a live autoscaler, and check sole ownership before removing a field from the manifest.

for a principal

Set a platform convention for field-manager names per automation and decide when a forced ownership transfer is allowed and who approves it.

What server-side apply adds

With kubectl apply --server-side, kubectl sends your manifest unchanged to the API server as an apply patch (application/apply-patch+yaml). The server merges it into the object. It also tracks, for every field, which actor declared or last wrote it. That record is metadata.managedFields, a list of entries, each with:

manager: the field manager name. kubectl uses kubectl for server-side apply and kubectl-client-side-apply for client-side updates. --field-manager overrides it. An apply request must carry one.
operation: Apply for declarative applies, Update for ordinary writes such as a PUT, a non-apply PATCH or a controller's status update.
apiVersion, time, fieldsType (FieldsV1) and fieldsV1, which is the set of owned field paths.
subresource: set when the write came through a subresource such as scale or status.

The ownership rules are what make conflicts possible:

An Apply that sets a field to a value different from another manager's owned value is rejected with HTTP 409 and the message Apply failed with 1 conflict: conflict with "<manager>" ...: .spec.replicas.
An Apply that sets the same value becomes a co-owner. That is not a conflict.
An Update never conflicts. It takes the field and removes it from the previous owners.
If an applier stops declaring a field it owns alone, the field is removed or reset to its default. If others still own it, it stays.

Diagnosing the portal's failure

The clinical-records portal runs records-portal on a 38-node spot and on-demand cluster. The manifest declares replicas: 7. During a 310 ms p99 latency spike, the HorizontalPodAutoscaler scaled it to 13 through the scale subresource. That was an Update, so the autoscaler's manager took spec.replicas away from kubectl. The next deploy applied replicas: 7, which differs from the owned value 13, so it is a conflict.

bash

kubectl apply --server-side -f records-portal.yaml
# Apply failed with 1 conflict: conflict with "<hpa-manager>" with subresource "scale" using apps/v1 at <time>: .spec.replicas

kubectl get deployment records-portal -o yaml --show-managed-fields
kubectl diff --server-side -f records-portal.yaml

Read the conflict message: it names the manager, the subresource, the apiVersion and the time of the write. Then inspect managedFields to see what else that manager owns. The with subresource "scale" part points straight at a scaler, not a person.

The resolution options

Option	Effect	When it is right
`--force-conflicts`	`kubectl` takes `replicas` and sets it to 7	Only when the other writer is wrong and has been stopped
Match the live value	Co-ownership, no change	A one-off to unblock, not a design
Remove the field from the manifest	The autoscaler keeps sole ownership	The right fix here
Separate field manager per concern	Clear ownership in shared objects	Several pipelines touching one object

--force-conflicts is tempting and wrong here. It cuts the portal from 13 to 7 Pods mid-spike, and the autoscaler scales back up on its next pass, so the two writers ping-pong on every deploy.

Removing replicas is safe in this state because kubectl no longer owns the field, so dropping it from the apply releases nothing. Check the ownership first. If kubectl were still the only owner (say, the autoscaler had not written yet), removing the line would release the field, and the Deployment would fall back to the default of 1 replica.

Migrating from client-side apply

When kubectl first runs --server-side on an object that was client-side applied, it migrates the kubectl-client-side-apply ownership and the last-applied annotation's fields into the kubectl manager. That way, fields you dropped earlier are still handled correctly. Use one manager name per pipeline and keep it stable. Renaming the manager makes the old name a separate owner, and your own old values then conflict with you.

Operating advice

Run kubectl diff --server-side or --dry-run=server in CI, so conflicts surface before the deploy step.
Treat --force-conflicts as a reviewed, one-time ownership transfer, not a pipeline default.
Give each automation its own --field-manager, so the conflict message tells you who disagreed.

## What server-side apply adds With `kubectl apply --server-side`, kubectl sends your manifest unchanged to the API server as an **apply patch** (`application/apply-patch+yaml`). The server merges it into the object. It also tracks, for every field, which actor declared or last wrote it. That record is **`metadata.managedFields`**, a list of entries, each with: - `manager`: the **field manager** name. kubectl uses `kubectl` for server-side apply and `kubectl-client-side-apply` for client-side updates. `--field-manager` overrides it. An apply request must carry one. - `operation`: `Apply` for declarative applies, `Update` for ordinary writes such as a PUT, a non-apply PATCH or a controller's status update. - `apiVersion`, `time`, `fieldsType` (`FieldsV1`) and `fieldsV1`, which is the set of owned field paths. - `subresource`: set when the write came through a subresource such as `scale` or `status`. The ownership rules are what make conflicts possible: 1. An **Apply** that sets a field to a value **different** from another manager's owned value is rejected with HTTP **409** and the message `Apply failed with 1 conflict: conflict with "<manager>" ...: .spec.replicas`. 2. An Apply that sets the **same** value becomes a **co-owner**. That is not a conflict. 3. An **Update** never conflicts. It takes the field and removes it from the previous owners. 4. If an applier stops declaring a field it owns **alone**, the field is removed or reset to its default. If others still own it, it stays. ## Diagnosing the portal's failure The clinical-records portal runs `records-portal` on a 38-node spot and on-demand cluster. The manifest declares `replicas: 7`. During a 310 ms p99 latency spike, the HorizontalPodAutoscaler scaled it to 13 through the `scale` subresource. That was an **Update**, so the autoscaler's manager took `spec.replicas` away from `kubectl`. The next deploy applied `replicas: 7`, which differs from the owned value 13, so it is a conflict. ```bash kubectl apply --server-side -f records-portal.yaml # Apply failed with 1 conflict: conflict with "<hpa-manager>" with subresource "scale" using apps/v1 at <time>: .spec.replicas kubectl get deployment records-portal -o yaml --show-managed-fields kubectl diff --server-side -f records-portal.yaml ``` Read the conflict message: it names the manager, the subresource, the apiVersion and the time of the write. Then inspect `managedFields` to see what else that manager owns. The `with subresource "scale"` part points straight at a scaler, not a person. ## The resolution options | Option | Effect | When it is right | |---|---|---| | `--force-conflicts` | `kubectl` takes `replicas` and sets it to 7 | Only when the other writer is wrong and has been stopped | | Match the live value | Co-ownership, no change | A one-off to unblock, not a design | | Remove the field from the manifest | The autoscaler keeps sole ownership | The right fix here | | Separate field manager per concern | Clear ownership in shared objects | Several pipelines touching one object | `--force-conflicts` is tempting and wrong here. It cuts the portal from 13 to 7 Pods mid-spike, and the autoscaler scales back up on its next pass, so the two writers ping-pong on every deploy. Removing `replicas` is safe **in this state** because `kubectl` no longer owns the field, so dropping it from the apply releases nothing. Check the ownership first. If `kubectl` were still the only owner (say, the autoscaler had not written yet), removing the line would release the field, and the Deployment would fall back to the default of 1 replica. ## Migrating from client-side apply When kubectl first runs `--server-side` on an object that was client-side applied, it migrates the `kubectl-client-side-apply` ownership and the last-applied annotation's fields into the `kubectl` manager. That way, fields you dropped earlier are still handled correctly. Use one manager name per pipeline and keep it stable. Renaming the manager makes the old name a separate owner, and your own old values then conflict with you. ## Operating advice - Run `kubectl diff --server-side` or `--dry-run=server` in CI, so conflicts surface before the deploy step. - Treat `--force-conflicts` as a reviewed, one-time ownership transfer, not a pipeline default. - Give each automation its own `--field-manager`, so the conflict message tells you *who* disagreed.

What exactly does `--force-conflicts` do to `managedFields`?
It sets `force=true` on the apply request. The server applies your values and moves ownership of every conflicting field to your manager, removing it from the other managers' sets. The other writer is not notified. A controller that still wants the field simply writes it again, so forcing only settles things if that writer has stopped.
Why would you give a pipeline its own `--field-manager` name instead of the default `kubectl`?
So that ownership is attributable. With every human and every pipeline applying as `kubectl`, they all co-own the same fields and silently undo each other. A distinct name such as `records-pipeline` makes a conflict name the actual disagreeing party, and it lets you see from `managedFields` which automation declared what. Keep the name stable, though: a renamed manager is a new owner.
The conflict names `kubectl-client-side-apply`. What does that tell you?
Someone, or an old job, is still running plain `kubectl apply` or `kubectl edit`-style updates against the object. Those writes are recorded under the client-side manager name. Find that writer and move it to server-side apply with the same manager as the pipeline, or retire it. Otherwise the two modes keep taking fields from each other.

saying these in an interview costs you the question

the conflict means someone else holds a lock on the Deployment
--force-conflicts is a safe default for CI pipelines
a controller's Update is rejected with a conflict just like an apply
managedFields is only informational and never affects writes
deleting a solely owned field from the manifest leaves it unchanged
any conflict can be fixed by retrying with a fresh resourceVersion

After a clinical-records portal's pipeline switched to `kubectl apply --server-side`, deploys fail with "Apply failed with 1 conflict" on the Kubernetes Deployment's `.spec.replicas`. How do you diagnose and resolve it?

§ answer

basics

solid answer

code

§ go deeper

▾level notes4 levels

▾deep dive~3 min

What server-side apply adds

Diagnosing the portal's failure

The resolution options

Migrating from client-side apply

Operating advice

▾common follow-ups3 questions

▾red flags6 flags

What server-side apply adds

Diagnosing the portal's failure

The resolution options

Migrating from client-side apply

Operating advice

answer

go deeper