Allowlists and Denylists Point in Opposite Directions
In one line
An AppProject is the boundary that decides "where this team may fetch from and where it may place things," and an ApplicationSet is the factory that stamps out Applications in bulk within that boundary. The place people get wrong most in AppProject is two fields that point in opposite directions: for cluster resources it is an allowlist (only what is listed is allowed), and for namespace resources it is a denylist (only what is listed is forbidden).
Why this was needed
When Argo CD has only one team, the default project is enough. The moment there are three teams, questions arise. What if team A accidentally deploys into team B's namespace? What if someone registers a personal GitHub repository as a source? What if one app creates a ClusterRoleBinding and makes itself cluster-admin? RBAC alone cannot stop this. Argo CD's service account has strong permissions anyway, and users borrow those permissions through Argo CD. So Argo CD filters once more at its own layer. That filter is the AppProject.
ApplicationSet came from a different kind of pain. With 20 services on five clusters, you have a hundred Applications. You cannot manage them by hand, and every time a new cluster joins you have to copy twenty of them. Generators turn this repetition into data.
How it works
You have to remember the key fields of AppProject together with their direction.
| Field | Direction | Meaning |
|---|---|---|
| sourceRepos | Allowlist | Only repositories matching the patterns listed here can be used as sources |
| destinations | Allowlist | You can deploy only to the clusters and namespaces listed here |
| clusterResourceWhitelist | Allowlist | Only the cluster-scoped kinds listed here can be created |
| namespaceResourceBlacklist | Denylist | Only the namespace-scoped kinds listed here cannot be created |
The third and fourth are opposite. For cluster resources, the default is to forbid everything, so if you list just Namespace, only that one opens. For namespace resources, the default is to allow everything, so if you list ResourceQuota, only that one closes. If you confuse this direction, you end up in a state where "what you thought you blocked is open," which is worse than having no policy.
syncWindows open and close sync by time of day. You only need to remember one rule. Deny beats allow. If a weekday business-hours allow overlaps with a weekend deny, the overlapping interval is forbidden. If you add manualSync: true, automated sync is blocked in that window and only a person pressing the button is allowed.
The basic ApplicationSet generators are list (a static list), clusters (registered clusters), git (directory scan or file reading), pullRequest (PR previews), and scmProvider (automatic discovery of an organization's repositories), and two kinds of combiners are added on top. matrix is the Cartesian product of two generators, so three clusters and four apps give twelve. merge joins with mergeKeys and overwrites the values of items that have the same key. You use merge when you lay down values common to all clusters and give only particular clusters something different.
The default template engine is fasttemplate, and it only does variable substitution. If you need conditions or loops, you have to turn on spec.goTemplate: true, and the moment you do, the reference syntax also changes to the form with a leading dot. If you give goTemplateOptions missingkey=error, a missing key raises an error instead of being silently turned into an empty string, which prevents the incident where an Application with an empty name gets created.
App-of-apps is a pattern in which an Application points at a directory containing other Applications. In exchange for the whole platform coming up with one bootstrap, deleting the parent makes all the children prune targets, so you must be especially careful with the parent's prune setting.
What it looks like in the field
The author's homelab has one cluster but divides tenants by namespace. What was learned here is that the boundary an AppProject draws and the actual resource boundary are different things. An AppProject's destinations is only a syntactic permission saying "you may deploy to capa-team-alpha," and it does not control at all how much of the node resources that namespace will consume. The resource boundary is made by ResourceQuota and LimitRange. And this is where the reason comes from that putting ResourceQuota in an AppProject's namespaceResourceBlacklist is the standard approach. If a tenant can raise its own quota by itself, it is not a quota. The quota is made by the platform team and the tenant cannot touch it, and that is what the combination of these two fields gives you.
In a cluster mixed with GPU nodes, I went one step further. The labels that GPU Feature Discovery attaches are strings, so comparison selectors such as "24GB or more" do not work, and so I attached meaning-based labels such as gpu.homelab/tier to the nodes myself and let workloads pick their own weight class. If you send such label values per tenant through an ApplicationSet's list generator, one template can stamp out apps that sit on different node groups.
What you will do in the next lab
In /root/capa-project/ you write one AppProject and two ApplicationSets (a list generator and a matrix generator), and then put the 3 tenant namespaces that project permits, each with its ResourceQuota, into a real cluster, to stand the syntactic boundary and the resource boundary side by side.