Authoring and Shipping Helm Charts
Three-Way Merge — What Gets Reverted and What Survives
Summary in one line
A Helm 3 upgrade builds a patch from three things — the old manifest, the new manifest, and the live objects in the cluster — so fields the chart declares are reverted and fields the chart does not know about survive.
Why this is needed
Incident response usually ends like this. Traffic surges and you urgently raise replicas with kubectl scale deploy/api --replicas=20, and later, to trace the cause, you attach a mark such as kubectl label deploy/api owner=ops. Things settle down, and a few days later a completely unrelated feature is deployed. And the next morning the replica count is back to 1, while the label is still attached.
If this asymmetry is not understood, Helm becomes "a tool that sometimes erases my changes." In reality it is one very clear rule.
A patch is built from three states
Helm 2 compared only the old manifest and the new manifest. So values a person changed in the cluster were not part of the comparison, and if there was no difference between the two, no patch went out, so the values the person changed stayed. Helm 3 adds the live objects in the cluster to this.
| Situation | Old manifest | New manifest | Live object | Result |
|---|---|---|---|---|
| Changed the replica count by hand | 1 | 1 | 5 | Reverts to 1 |
| Added a label by hand | none | none | present | Stays as it is |
| Removed a label from the chart | present | none | present | Gets deleted |
The first row is where it differs from Helm 2. If the chart declares replicas, the owner of that field is the chart, and if the live object differs, it is brought in line with the declaration. The second row is not included in the patch because the chart does not know that field. The third row is in the old manifest and not in the new one, so it is read as meaning "delete it."
A practical conclusion follows from this — do not declare in the chart a replicas that an autoscaler manages. If you declare it, on every deployment Helm reverts the value the autoscaler decided, and a tug-of-war arises in which the autoscaler raises it again. The standard approach is to leave replicas out of the chart and leave it to the HPA.
Why values do not carry over
The second trap is on the side of values, not objects.
helm upgrade api ./api --set replicas=4 --set logLevel=debug # 리비전 2
helm upgrade api ./api --set replicas=6 # 리비전 3
The user values of revision 3 are only {replicas: 6}. logLevel is gone and returns to the chart default. This is not a bug but the default behavior — an upgrade recomputes the release from "the values given this time." You can check with helm get values api.
There is one exception, and it confuses people the most. If you give no values at all (neither --set nor -f), Helm uses the previous release's user values as they are. So if you run only helm upgrade api ./api, last time's values are kept, and the moment you give even one --set, all the rest fall away. This is the spot where the intuition "if I leave a value out it will go back to the default" works in exactly the opposite way. To discard last time's values for certain, state --reset-values explicitly.
There are three options for carrying values over, and their similar names make them confusing.
| Option | What it does |
|---|---|
--reuse-values |
Carries over the previous release's values as they are and layers only this time's --set on top |
--reset-values |
Discards last time's values and layers only this time's on top of the chart defaults (same as the default behavior) |
--reset-then-reuse-values |
Goes back to the chart defaults, layers last time's user values on again, and then layers this time's |
The difference between --reuse-values and --reset-then-reuse-values shows when the chart defaults have changed. The former carries over the values the previous release computed wholesale, so the chart's new defaults get buried, and the latter lays the new defaults down first and then layers on only what the user stated explicitly, so the new defaults are reflected. If you raised the defaults along with the chart, this difference decides the deployment result.
--force is replacement, not a patch
--force replaces the object instead of patching. It is an escape hatch for when you hit an immutable field that a patch cannot change, such as a Service's clusterIP, but because it is a replacement, the object in question briefly disappears and is created again. For a Deployment, all Pods are recreated, and for a Service, routing is cut off for a moment.
One more thing comes with replacement. Because it swaps in the whole new manifest, even the "fields the chart does not know about" that survived in ordinary upgrades vanish. An owner label attached by hand normally stays in an ordinary upgrade, but after --force it is gone. "The upgrade isn't taking, so force it for now" is one of the most expensive habits in production.
What it looks like in the field
The most common accident is one --set dropping out of a deployment script. When there are about eight values, someone deletes one line, and only that value quietly returns to its default. No error appears and the deployment is shown as a success. The standard solution to this problem is not the carry-over options but keeping the values in the repository as a file. With a single line of -f values-prod.yaml, what you deployed with stays in the commit history, goes through code review, and can be read by the next person. --reuse-values goes in the opposite direction — it is convenient, but the values remain only inside the release and are not visible anywhere in the repository.
If you want to see what will change before deploying, there is helm upgrade --dry-run=server. It sends the render result to the real API server, even validating it, and does not create a release. But this is "validation," not "seeing the difference." To see line by line what changes, you need a plugin such as helm-diff, and it is not in this lab environment.
What you will do in the next lab
You deploy a Deployment, change the replica count and a label by hand in the cluster, and then run an upgrade that touches nothing in the chart to see what reverts and what remains. You see that a label removed from the chart is also deleted in the cluster, then move on to the values side and compare, through the release records, how a default upgrade discards the previous user values and how the three carry-over options differ. At the end you use the server-side preview and --force once each and sort out the rules in a report.