My Value Keeps Reverting: Finding the Field's Owner
Goal
On the kwok cluster, you create two managers with server-side apply and read managedFields directly. You trigger with real objects the conflict and force, the letting go of fields, and even the conflict on the scale subresource, and at the end you connect it with Argo CD's ignore rule.
Why it matters
Even if you turn on GitOps, the subject that writes the cluster's objects is not just one. Argo CD writes, an autoscaler writes replicas, a webhook inserts a container, and a person writes with kubectl when in a hurry. Server-side apply is a device by which the API server holds, as a record, "who is this field's owner". Only if you can read this record can you answer three questions — why is my apply rejected, why, if I push it in by force, is the other side rejected next time, and why, when I let go of a value, does it not return to the old value. And this record connects to the Argo CD side. This is because the managedFieldsManagers of the ignore rule picks what to compare by this ownership record, not by field path.
Steps
- On the kwok cluster, create the namespace
ga-ssa, and in/root/ga-ssa/deploy.yaml, write a Deploymentweb—spec.replicas2, the selector and label areapp: web, and one container (web, imagenginx:1.25). Apply it withkubectl apply --server-side --field-manager=gitops. - Read the ownership record with
kubectl -n ga-ssa get deploy web -o json --show-managed-fieldsand save it, one line per manager in the form<관리자> | <연산>(manager, then operation), to/root/ga-ssa/managers.txt.gitopsmust be in it with theApplyoperation. - In
/root/ga-ssa/hotfix.yaml, write the same Deployment but withspec.replicasat 5 and the image atnginx:1.27. Try a server-side apply with--field-manager=hotfix, and save the failure output, including standard error, to/root/ga-ssa/conflict.txt. You do not use--force-conflictsyet. - Do the same apply again together with
--force-conflictsand save the success output to/root/ga-ssa/force.txt. Now the cluster's image isnginx:1.27, and the owner of that field ishotfix. - Create
/root/ga-ssa/hotfix-slim.yaml— the same ashotfix.yamlbut without thespec.replicasline. After a server-side apply with--field-manager=hotfix, save the list of managers that ownspec.replicasto/root/ga-ssa/release.txt(an empty file if there are none). Check what the cluster's replicas becomes. - In
/root/ga-ssa/web-csa.yaml, write a Deploymentweb-csa(same namespace, the label and container name areweb-csa, imagenginx:1.25) and put it up, without--server-side, with a plainkubectl apply. For the two Deployments (webandweb-csa), save the list ofmetadata.annotationskeys to/root/ga-ssa/csa.txt. - In
/root/ga-ssa/scale-deploy.yaml, write a Deploymentweb-scale(2 replicas, label and container nameweb-scale, imagenginx:1.25) and apply it server-side with--field-manager=gitops. Then raise it withkubectl -n ga-ssa scale deploy web-scale --replicas=4, and try applying the same file again with the same manager name. Save the failure output to/root/ga-ssa/scale-conflict.txt. - Save to
/root/ga-ssa/live.yamlthe cluster'swebas YAML including the ownership record (-o yaml --show-managed-fields). In/root/ga-ssa/argocd-cm.yaml, put the keyresource.customizations.ignoreDifferences.apps_Deploymentand inmanagedFieldsManagers, writehotfix, and injsonPointers,/spec/replicas. Save the output ofargocd admin settings resource-overrides ignore-differences /root/ga-ssa/live.yaml --argocd-cm-path /root/ga-ssa/argocd-cm.yamlto/root/ga-ssa/ignore.txt.
Notes
kubectl get ... -o jsonhides managedFields — add--show-managed-fields.- You decide the manager name with
--field-manager. The default differs by command. - The conflict message tells you which field belongs to whom. Read that list first.
- Common mistake: attaching
--force-conflictsout of habit — the ownership moves and the other side gets blocked next time. - Common mistake: expecting that if you remove a field from the manifest, it goes back to the old value. It becomes the API default.
- Reference: https://kubernetes.io/docs/reference/using-api/server-side-apply/
State your name and apply
On the kwok cluster, create the namespace ga-ssa, and in /root/ga-ssa/deploy.yaml, write a Deployment web — spec.replicas 2, the selector and label are app: web, and one container (web, image nginx:1.25). Apply it with kubectl apply --server-side --field-manager=gitops.
--field-manager is the name that tells the API server "who made this change". Server-side apply records which fields are owned, per this name — a concept that client-side apply does not have.
Read the ownership record
Read the ownership record with kubectl -n ga-ssa get deploy web -o json --show-managed-fields and save it, one line per manager in the form <관리자> | <연산> (manager, then operation), to /root/ga-ssa/managers.txt. gitops must be in it with the Apply operation.
If you leave out this option, .metadata.managedFields comes out as null — because kubectl hides it by default. Go through the array with jq and pull out manager and operation.
When a second manager tries to write the same field
In /root/ga-ssa/hotfix.yaml, write the same Deployment but with spec.replicas at 5 and the image at nginx:1.27. Try a server-side apply with --field-manager=hotfix, and save the failure output, including standard error, to /root/ga-ssa/conflict.txt. You do not use --force-conflicts yet.
The server rejects, saying "someone else holds this field", and tells you in a list which fields. Without this rejection, two automations would quietly overwrite each other's values. Be careful that your answer sheet does not stop with the failure.
Take ownership by force
Do the same apply again together with --force-conflicts and save the success output to /root/ga-ssa/force.txt. Now the cluster's image is nginx:1.27, and the owner of that field is hotfix.
Force does not remove the conflict; it moves the ownership. The old owner drops out of that field, so the next time the old owner applies its own value again, this time it meets the conflict — if two automations use force by turns, it becomes an endless fight.
If you let go of a field, the value does not go back to the owner
Create /root/ga-ssa/hotfix-slim.yaml — the same as hotfix.yaml but without the spec.replicas line. After a server-side apply with --field-manager=hotfix, save the list of managers that own spec.replicas to /root/ga-ssa/release.txt (an empty file if there are none). Check what the cluster's replicas becomes.
If you remove a field from the manifest, that manager lets go of that field in the ownership list. But the value does not return to the old owner's value — a field nobody owns becomes the API default. For the owner list, with jq you can pick the entries where .fieldsV1."f:spec"."f:replicas" is not null.
What client-side apply leaves behind
In /root/ga-ssa/web-csa.yaml, write a Deployment web-csa (same namespace, the label and container name are web-csa, image nginx:1.25) and put it up, without --server-side, with a plain kubectl apply. For the two Deployments (web and web-csa), save the list of metadata.annotations keys to /root/ga-ssa/csa.txt.
Client-side apply puts "the manifest I sent last time" whole into one annotation and compares against it to decide what to delete. So there is no concept of an owner, and when two tools handle the same object, they delete each other's fields. The one put up server-side does not have that annotation.
kubectl scale comes in through a different door
In /root/ga-ssa/scale-deploy.yaml, write a Deployment web-scale (2 replicas, label and container name web-scale, image nginx:1.25) and apply it server-side with --field-manager=gitops. Then raise it with kubectl -n ga-ssa scale deploy web-scale --replicas=4, and try applying the same file again with the same manager name. Save the failure output to /root/ga-ssa/scale-conflict.txt.
kubectl scale writes not to the whole object but to a subresource called scale. So the manager kubectl and the subresource scale are recorded separately in the ownership record, and the conflict message also tells you that fact. The conflict an autoscaler creates has exactly this shape.
Remove from the comparison by owner name
Save to /root/ga-ssa/live.yaml the cluster's web as YAML including the ownership record (-o yaml --show-managed-fields). In /root/ga-ssa/argocd-cm.yaml, put the key resource.customizations.ignoreDifferences.apps_Deployment and in managedFieldsManagers, write hotfix, and in jsonPointers, /spec/replicas. Save the output of argocd admin settings resource-overrides ignore-differences /root/ga-ssa/live.yaml --argocd-cm-path /root/ga-ssa/argocd-cm.yaml to /root/ga-ssa/ignore.txt.
These two are the ways to express in Argo CD "do not compare fields the autoscaler wrote" — remove by path or remove by manager name. However, the preview command renders only path rules, so what the manager-name rule will actually filter, a person must read the ownership record to judge.