CGOA — GitOps Certified Associate
Two controllers rewriting the same setting
In one line
If you directly edit an Application that an ApplicationSet created, the parent may reconcile it back. To stop the work, you must first check who owns that field.
Why this was needed
You turned off automated sync to investigate an incident. You confirmed on the screen that it was off and stepped away for a moment, and it is back on. You ask a colleague, thinking they turned it on, but they say nobody touched it. Restarting the controller or pressing the same button several times cannot explain the problem. That is because the thing whose setting you changed may not have been the top-level source.
GitOps's reconcile loop is not just one. The Application controller matches the Git target that an Application specifies to the actual resources. The ApplicationSet controller matches the target of an Application created from a generator and a template to the actual Application object. If an operator changes an automation field on a child Application, the second loop may see it as drift and restore it. Fixing a ConfigMap read from Git and fixing the Application itself are changes at different layers.
How it works
In this lab, the list generator holds only one item, alpha. It is not a lab about increasing the number of services but about narrowing the ownership relationship for observation. The ApplicationSet template creates alpha's Application, and that Application reads the apps/alpha directory of the internal Git and creates a small ConfigMap. As a point of comparison, an independent Application and a separate ConfigMap are also in place. If the independent target changes along with it, you have drawn the scope wrongly.
| Reconciler | Where it reads the target | What it changes | The question to ask during maintenance |
|---|---|---|---|
| ApplicationSet controller | The generator and template | The Application's settings | Who is rewriting this automation value? |
| Application controller | The Git the Application specifies | Actual resources such as a ConfigMap | Which Git commit is actually being applied? |
ownerReferences is an important clue to the first relationship. If you check kind, name, uid, and controller in the child Application's ownerReferences, you can tell which ApplicationSet it is linked to. If you compare only names, you can miss a parent recreated with the same name, so cross-check the UID as well. A ConfigMap is told apart by reading Argo CD's tracking annotation to see which Application manages it. Do not treat these two kinds of ownership information as the same thing.
The value that explicitly turns off automated sync is false in spec.syncPolicy.automated.enabled. You must not read null or a missing field as the same as false. Even if prune and selfHeal are true, if enabled=false, automated sync is off. This environment always states enabled explicitly as a boolean to avoid guessing at defaults.
That the API accepted the change when enabled=false was written on the child and that the state is maintained are also different things. The lab first preserves the false and the resourceVersion from the patch response, and in a later observation confirms that enabled of the same Application UID has returned to true. This is so that a request in which false was not reflected from the start is not called a restoration experiment. resourceVersion is an opaque string for telling observation versions apart and is not used as a number for computing time or a global sequence number.
If you declare enabled=false in the parent template, the target of the children to be created itself changes. The child's false is no longer a difference from the parent, so it can be maintained. A template may apply in common to several children, so in a real team environment you must first check the generator result and the list of affected Applications before making a change. The lab limits that scope to one child, but this fact is not generalized to mean that only one changes in large-scale operations.
What it looks like in the field
Consider an organization that deploys automatically to the development environment and deploys after manual confirmation to the verification environment. If you apply one template to all environments and then turn it off with a button only in the verification environment, the source and the actual state conflict. The operating practice the team wants to guarantee must be expressed as a template or an explicit exception policy. A procedure in which someone remembers to turn it off again every time is not a declaration of the desired state but repeated manual work.
When investigating, you read documents at different layers, as follows. All of them are commands to run inside the personal lab VM.
kubectl -n argocd get applicationset cgoa-appset-ownership -o json
kubectl -n argocd get application cgoa-appset-ownership-alpha -o json
kubectl -n cgoa-appset-ownership get configmap alpha-config -o json
In the first document, look at the template and the generator; in the second, the ownerReferences and the automation policy; and in the third, the actual data. From a single line saying automation is off, you cannot tell which reconcile loop has stopped.
What you will do in the next lab
First you compare the record of the API accepting the child's change with the record of the parent reverting it. Next, you fix the parent template and look separately at the child's settings and whether the Git change was applied. The ApplicationSet itself is applied by the helper from the lab's bootstrap declaration file, and a third loop in which a higher-level Application also reconciles that from Git is not included in this scope. You must make this boundary explicit so you do not confuse the declaration applied directly in the lab with the full GitOps wiring of production.
Official documentation
- ApplicationSet resource modification control: Distinguish child changes from exception rules.
- Automated sync: Check the meaning of true, false, and null for enabled.