Open Up a Release — Where It Is Stored, the Retention Window, Bringing It Back

Goal

You decode for yourself where and in what form a Helm release is stored in the cluster, and go one full cycle through how narrow the retention window is and how to revive a deleted release.

Why it matters

What helm rollback can and cannot undo all comes from the storage structure. A release is a single Secret in a namespace, and inside it the rendered manifest is compressed whole. So what was written in the manifest comes back, and what happened outside does not.

Two more things follow from the same structure. A release name is unique only within a namespace, and there is an upper limit on the number of revisions kept. This is why "rolling back to half a year ago" is usually impossible. The habit of first checking "what values did it come up with?" with helm get values when an incident occurs also sticks only once you know this structure.

Environment

This Pod brings up a real kube-apiserver with kwok. Pods do not actually run, but release Secrets, revisions, and object changes are all real. So grading is also done not from files but by asking helm and the apiserver again. The working directory is /root/hs-rel and the outputs go under /root/hs-rel/out.

Steps

1. Create a chart in /root/hs-rel and install it as shop in rel-a.
2. Decode the release Secret of revision 1 and save it to /root/hs-rel/out/release-v1.json.
3. Install the same name shop in rel-b as well.
4. Upgrade four times with --history-max 3 and watch old revisions disappear.
5. Save the results of helm get values and helm get manifest to /root/hs-rel/out.
6. Delete the release in rel-b with --keep-history and save the record.
7. Revive the deleted release with a rollback.
8. Summarize it in seven lines in /root/hs-rel/out/report.txt.

Notes

• When decoding the Secret, base64 -d is applied twice. Kubernetes adds one layer, and helm adds another layer of encoding.
• By default, helm list shows only deployed. To see deleted or stuck ones, add -a.
• A common mistake is leaving out --keep-history in step 6. Then there is nothing to revive in step 7.
• Another is receiving with --all in step 5. Then the chart defaults get mixed in and you cannot tell "what a person put in."

Create a chart and deploy it under a name

In /root/hs-rel, create a chart with helm create webapp and install it in the rel-a namespace under the name shop. Create the namespace together with --create-namespace.

It is helm install <릴리스이름> <차트경로> -n <네임스페이스> --create-namespace (release name, chart path, and namespace in the placeholders). When the installation finishes, check the name of the created Deployment with kubectl -n rel-a get deploy. The release name is put in front, not a name the chart decided. This cluster is kwok, so Pods do not actually run, but the releases and objects are all real.

Decode the Secret where the release is stored

Pick the one for revision 1 among the helm release Secrets in rel-a, decode its content, and save the whole release JSON to /root/hs-rel/out/release-v1.json.

Check the name with kubectl -n rel-a get secret -l owner=helm,name=shop. Pull out the value with -o jsonpath='{.data.release}', then do base64 -d twice and gzip -d, and the release JSON comes out. This is because Kubernetes encoded it in one layer and helm in another. The rendered YAML is in the .manifest field of that JSON as a string, so this time save the JSON as a whole.

One more release with the same name in a different namespace

Install the same chart in the rel-b namespace, also under the name shop. The two releases must live side by side without conflict.

A release name is unique not cluster-wide but only within a namespace. This is because the record goes into that namespace's Secret. See everything with helm list -A, and check where the record was created with kubectl -n rel-b get secret -l owner=helm.

Shrink the retention window and watch old revisions disappear

Upgrade shop in rel-a four times with --history-max 3. In the last upgrade, replicaCount must be 5.

Run helm upgrade shop ./webapp -n rel-a --set replicaCount=<값> --history-max 3 four times with replicaCount 2, 3, 4, and 5 (the value goes in the placeholder). Then look at helm history shop -n rel-a and kubectl -n rel-a get secret -l owner=helm,name=shop side by side. The revision number climbs to 5, but only three remain. The point of this step is that the window you can go back to is narrower than you think.

Pull out the values and manifest actually put into this release

Save helm get values as JSON to /root/hs-rel/out/values.json and the result of helm get manifest to /root/hs-rel/out/manifest.yaml. The values must contain only what the user gave.

helm get values shop -n rel-a -o json shows only the values the user gave, and adding --all shows them merged with the chart defaults. What you look at first in an incident investigation is the former. helm get manifest is the YAML that release actually applied, so the replicas written there and the replicas that kubectl get deploy answers should be the same.

Delete it while keeping the record

Delete shop in rel-b with --keep-history, and save the output of helm list -n rel-b -a and helm history shop -n rel-b together to /root/hs-rel/out/uninstalled.txt.

It is helm uninstall shop -n rel-b --keep-history. If you just run helm uninstall, even the release Secrets disappear and it cannot be revived. After deleting, helm list -n rel-b shows nothing, but if you add -a you can see the one remaining in the uninstalled state. Confirm with your own eyes that the default of helm list hides deleted releases.

Revive the deleted release

Roll back shop in rel-b to revision 1 to revive it. Even after reviving it, the uninstalled revision must remain in the record.

It is helm rollback shop 1 -n rel-b. Because you kept the record, you can come back without installing again. If you run helm install again, the history starts anew from number 1 and what happened is lost. If you look at helm history after the rollback, a new revision with 'Rollback to 1' written on it is attached above the uninstalled revision.

Summarize the seven steps in numbers

Write seven lines in /root/hs-rel/out/report.txt. They are RELEASES_TOTAL, STORAGE, HISTORY_MAX, REL_A_KEPT, REL_A_LATEST, RESOURCE_PREFIX, and SCOPE.

Do not guess the values; count them from the cluster and write them. Check how many shop releases there are with helm list -A -a, and how many revisions remain and what the largest number is with helm history shop -n rel-a. For STORAGE, write in one word where helm 3 keeps the release, and for SCOPE, how far the release name is unique. RESOURCE_PREFIX is the Deployment name you saw in step 1.