TT Lab
Get started
Learn Learning paths Courses

GitOps and Argo CD

One Schemaless ConfigMap Decides the Platform

Continue in TT Lab

In one sentence

argocd-cm is a ConfigMap with no schema, so a typo becomes not an error but "no configuration", and argocd admin settings validate is not a tool that checks it but a tool that hands back the values it read.

Why this check was needed

When you start managing Argo CD by declaration, a single argocd-cm grows. Accounts, SSO, kustomize build options, kinds not to watch, the ownership-marking method, health rules, ignore rules, and resource actions all come in as keys of this file. The problem is that this file is an ordinary ConfigMap. Kubernetes does not check key names. Even if you write kustomize.buildOptions as kustomize.buildOption, the apply succeeds, and Argo CD does not know that key, so it just passes over it. There is no warning anywhere on the screen.

This kind of mistake lives a very long time. That an option is not taking effect is usually discovered by accident while doing something else, and by then nobody remembers who put that line in or why.

How it works

argocd admin settings validate reads argocd-cm (and argocd-secret if needed), divides it into five sections, and prints the interpreted result.

✅ accounts            3 accounts
✅ general             Dex is configured
✅ kustomize           --enable-helm
✅ repositories        1 repositories
✅ resource-overrides  2 resource overrides

How to read it matters here. ✅ means "reading that section did not fail", not "the configuration is right". The real information is on the line below it. If you added two accounts and the number stays the same, the key name is wrong, and if you put in a build option and default options comes out, that option does not exist. That is, this command is not a linter but a read-back. It is worth using only for seeing whether the values I put in come back out.

And there is a limit. Not every setting appears in these five sections. resource.exclusions and resource.inclusions are not summarized in any section. For such keys, the only way is to parse the file directly to check. inclusions needs special care — the moment you write even one, every kind not listed there becomes invisible.

The ownership-marking setting is also worth knowing. If application.resourceTrackingMethod is label, Argo CD writes the app name into a label such as app.kubernetes.io/instance. But a label value cannot exceed 63 characters. As the organization grows and app names get long, you hit this limit, and because of truncated names, different apps end up with the same ownership mark — this is where the incident of one app's synchronization touching another app's resources comes from. The annotation method writes into the argocd.argoproj.io/tracking-id annotation, so there is no length limit, and that is why it is now the recommended method.

Repositories also have two generations mixed together. The repositories list in argocd-cm is the old way, and now you keep one Secret per repository and attach the label argocd.argoproj.io/secret-type: repository. The big difference is that it can hold credentials together. There is a quiet failure here too — if you leave out the label, Argo CD cannot see that Secret. No error occurs.

What you see in the field

The most common scene is the inquiry "the kustomize build that uses Helm doesn't work". They say they put in kustomize.buildOptions, but the actual file says buildOption. It is a job that one run of this command finishes in 3 seconds, but without it you spend half a day going back and forth between controller logs and Pod restarts.

The second is touching the exclusion list wrongly. To reduce load, you write only a few kinds in resource.inclusions, and all the other resources vanish from the app tree. People panic, thinking the resources were deleted. This setting does not appear in the validate output, so before changing it you must read the file directly and count what it leaves.

The third is migration. If you switch from the label method to the annotation method, Argo CD may fail to recognize resources carrying the old label as its own. So this change is not one to do quietly but one where you leave the read values before and after the change as files and attach them to the review. The last step of this lab is exactly that habit.

The limits of this lab environment

The lab Pod has no Argo CD controller or server. So you cannot change a setting and see the screen change, nor log in with SSO. Instead, the code that actually interprets the configuration is inside the CLI, so you can confirm with exactly the same result what was read and what was ignored. The repository Secret and the ownership-mark annotation you actually put up on the kwok cluster.

What you will do in the next lab

Starting from the read-back of an empty configuration, you add accounts, build options, an exclusion list, ownership marking, and a repository one item at a time. Midway, you deliberately get a key name wrong by one character and see how it quietly disappears. You run into the 63-character label limit directly on the kwok cluster, and at the end you leave the read-back of the beginning and the end as a diff.