We deployed tag v1, and v2 showed up

Goal

You set up Argo CD Applications that track a branch, a tag, and a commit SHA respectively, and confirm from the actual sync records which names move and which do not. Then you see why a rollback in GitOps has to be a new commit.

Why it matters

The second OpenGitOps principle says the desired state must be stored "versioned and immutable." A Git commit is a hash of its content and does not change, but branches and tags are merely labels that point to a commit, so anyone can move them. targetRevision: v1 means "what the name v1 points to right now," not "the commit we originally reviewed." If you do not know this difference, unreviewed content gets deployed under a release name that passed review, and the audit record still says v1. Rolling back comes from the same principle. If you turn only the cluster back to an old state, Git and the cluster split apart, and if you erase Git's history, the evidence of what was deployed and when disappears.

Steps

Create the bare repository /srv/bare/rev.git and clone it to /root/cgoa-rev/repo. Commit a ConfigMap release (do not write a namespace, data version: v1) to app/release.yaml and push to main, then tag that commit v1 and push the tag too. Write the 40-character SHA of that commit on one line in /root/cgoa-rev/v1.sha.
Write and apply three Applications in /root/cgoa-rev/apps.yaml. The names and target namespaces are rev-branch, rev-tag, and rev-sha, the source is the app path of git://gitd.gitsrv.svc.cluster.local:9418/rev.git, and the targetRevision is, in order, main, v1, and the 40-character SHA written in v1.sha. For all three, use project default, turn on automated sync (prune and selfHeal) and CreateNamespace=true, and check that all three apps become Synced.
In /root/cgoa-rev/repo, change the version in app/release.yaml to v2, commit, and push to main (do not touch the tag). After hard refreshing the three apps, write each app's status.sync.revision into /root/cgoa-rev/after-commit.json in the form {"branch": ..., "tag": ..., "sha": ...}. The branch must go to the new commit, and the tag and sha must stay on the v1 commit.
Forcibly move the tag v1 to main's latest commit (v2) and run git push -f, then hard refresh rev-tag. Write the status.history[].revision of rev-tag, in order, into /root/cgoa-rev/tag-history.json as {"target": "v1", "revisions": [...]}. Two different commits must be recorded under the same name v1, the ConfigMap in the rev-tag namespace must change to v2, and rev-sha must stay at v1.
First, move the tag v1 back to the original commit of v1.sha and push, and make rev-tag sync to that commit again. Then place an executable hook at /srv/bare/rev.git/hooks/pre-receive that rejects a push that changes or deletes an existing refs/tags/* and allows creating new tags and updating branches. Finally, try a push that moves v1 again, and save the entire rejected output to /root/cgoa-rev/guard.txt.
main is now v2. Do not erase history; use git revert to create a new commit that reverts the v2 commit, and push it. rev-branch must sync to that new commit and the ConfigMap must go back to v1. In /root/cgoa-rev/rollback.json, write revert_sha (the new main SHA), deployed (the version value of the ConfigMap in the rev-branch namespace), and equals_v1_sha (whether the new SHA equals v1.sha, a boolean).
With automated sync left on, run argocd app rollback rev-branch 0 --core and save the entire output, including the rejection message, to /root/cgoa-rev/rollback-refused.txt. Core mode looks up the Argo CD configuration in the kubeconfig's current namespace, so copy the k3s kubeconfig to /root/cgoa-rev/kubeconfig, change its current namespace to argocd, and run it by pointing KUBECONFIG at it. Do not turn off automated sync just to make the rollback succeed. When you are done, rev-branch must still be Synced on main's latest commit.
In /root/cgoa-rev/report.json, write mutable_refs (the two kinds of refs that moved, as branch and tag, an array), immutable_ref (commit-sha), pinned_revision (the current status.sync.revision of rev-sha), tag_guard (pre-receive), and rollback (git-revert). The grader re-checks the current state of the three apps along with the file values.

Notes

The VM has k3s, Argo CD v3.5.2, and an in-cluster git daemon (gitd.gitsrv). Every bare repository under /srv/bare appears as git://gitd.gitsrv.svc.cluster.local:9418/<이름>.git (the placeholder is the name).
To avoid waiting for a new commit, use kubectl -n argocd annotate app <이름> argocd.argoproj.io/refresh=hard --overwrite (the placeholder is the app name).
Reading the status: kubectl -n argocd get app -o custom-columns=N:.metadata.name,S:.status.sync.status,R:.status.sync.revision
Common mistake: moving the tag only locally and forgetting git push -f origin v1. The remote tag stays as it was, so nothing happens.
Common mistake: trying to put the hook in first in step 5 and then move the tag back. The hook rejects that push too.
OpenGitOps principles · Argo CD tracking strategies · argocd app rollback · githooks

Tag the v1 commit and write down the SHA

Create the bare repository /srv/bare/rev.git and clone it to /root/cgoa-rev/repo. Commit a ConfigMap release (do not write a namespace, data version: v1) to app/release.yaml and push to main, then tag that commit v1 and push the tag too. Write the 40-character SHA of that commit on one line in /root/cgoa-rev/v1.sha.

You create an empty bare repository with git init --bare. The git daemon exports /srv/bare as is, so there is nothing separate to register. A tag must be pushed separately from the branch for it to exist on the remote. You get the SHA with git rev-parse.

Three apps tracking a branch, a tag, and a SHA

Write and apply three Applications in /root/cgoa-rev/apps.yaml. The names and target namespaces are rev-branch, rev-tag, and rev-sha, the source is the app path of git://gitd.gitsrv.svc.cluster.local:9418/rev.git, and the targetRevision is, in order, main, v1, and the 40-character SHA written in v1.sha. For all three, use project default, turn on automated sync (prune and selfHeal) and CreateNamespace=true, and check that all three apps become Synced.

The three apps differ only in the one targetRevision line. For now all three names point to the same commit, so status.sync.revision must be the same too.

When v2 lands on main, who follows?

In /root/cgoa-rev/repo, change the version in app/release.yaml to v2, commit, and push to main (do not touch the tag). After hard refreshing the three apps, write each app's status.sync.revision into /root/cgoa-rev/after-commit.json in the form {"branch": ..., "tag": ..., "sha": ...}. The branch must go to the new commit, and the tag and sha must stay on the v1 commit.

The way to request a refresh is the annotation argocd.argoproj.io/refresh=hard. Do not guess the values to write in the file; read them from the Application status.

We deployed the v1 tag, but v2 came up

Forcibly move the tag v1 to main's latest commit (v2) and run git push -f, then hard refresh rev-tag. Write the status.history[].revision of rev-tag, in order, into /root/cgoa-rev/tag-history.json as {"target": "v1", "revisions": [...]}. Two different commits must be recorded under the same name v1, the ConfigMap in the rev-tag namespace must change to v2, and rev-sha must stay at v1.

A tag is just a name, not a commit. Each entry in history keeps both the source.targetRevision at that time and the actual revision.

Block moving a tag that already exists

First, move the tag v1 back to the original commit of v1.sha and push, and make rev-tag sync to that commit again. Then place an executable hook at /srv/bare/rev.git/hooks/pre-receive that rejects a push that changes or deletes an existing refs/tags/* and allows creating new tags and updating branches. Finally, try a push that moves v1 again, and save the entire rejected output to /root/cgoa-rev/guard.txt.

pre-receive receives lines of 옛SHA 새SHA 참조이름 on standard input (the three fields are the old SHA, the new SHA, and the ref name). For a ref being created, the old SHA is 40 zeros, and for a ref being deleted, the new SHA is too. If the hook exits with a non-zero value, the whole push is rejected. Do the push that moves the tag back before you put the hook in place.

A rollback is a new commit

main is now v2. Do not erase history; use git revert to create a new commit that reverts the v2 commit, and push it. rev-branch must sync to that new commit and the ConfigMap must go back to v1. In /root/cgoa-rev/rollback.json, write revert_sha (the new main SHA), deployed (the version value of the ConfigMap in the rev-branch namespace), and equals_v1_sha (whether the new SHA equals v1.sha, a boolean).

A reset followed by a force-push also makes the content v1, but the v2 commit vanishes from main's history. Choose the approach that keeps a record of who reverted what and when.

Why a rollback on an automated-sync app is rejected

With automated sync left on, run argocd app rollback rev-branch 0 --core and save the entire output, including the rejection message, to /root/cgoa-rev/rollback-refused.txt. Core mode looks up the Argo CD configuration in the kubeconfig's current namespace, so copy the k3s kubeconfig to /root/cgoa-rev/kubeconfig, change its current namespace to argocd, and run it by pointing KUBECONFIG at it. Do not turn off automated sync just to make the rollback succeed. When you are done, rev-branch must still be Synced on main's latest commit.

Argo CD does not roll an app with automated sync turned on back to an old revision, because even if it did, the next reconciliation would return it to Git. kubectl config set-context --current --namespace=... changes only the kubeconfig file you point it at.

Report the names that change and the names that do not

In /root/cgoa-rev/report.json, write mutable_refs (the two kinds of refs that moved, as branch and tag, an array), immutable_ref (commit-sha), pinned_revision (the current status.sync.revision of rev-sha), tag_guard (pre-receive), and rollback (git-revert). The grader re-checks the current state of the three apps along with the file values.

Recall what actually moved in the earlier steps and which app never moved to the end. Read pinned_revision from the status.