Canary, Blue-Green and Rolling Back

Goal

Write canary and blue-green Rollout manifests and an AnalysisTemplate responsible for metric-based automatic rollback, put the corresponding Services and Deployment into the cluster, and perform a rollout and a rollback by hand.

Why it matters

The core of progressive delivery is two things: "how much of the new version to expose" and "what to watch to decide whether to continue." setWeight is the answer to the first and analysis is the answer to the second. A pause without a duration is the place where that judgment is handed to a person, and an AnalysisTemplate is the place where the same judgment is handed to metrics. Meanwhile, rollback itself is not an invention of Argo Rollouts; it is possible because Kubernetes keeps ReplicaSets as revisions. If you touch both layers in this lab, you can tell how far a Rollout is actually new.

Steps

Create the /root/capa-rollout/ directory and, in rollout.yaml, write apiVersion argoproj.io/v1alpha1, kind Rollout, metadata.name capa-web, metadata.namespace capa-rollout, spec.replicas 4, spec.selector.matchLabels.app capa-web, and the container image nginx:1.27.
Make spec.strategy.canary.steps in the same file five or more, with the first step setWeight: 20, the second pause: {duration: 30s}, the third setWeight: 50, and the last step setWeight: 100.
In /root/capa-rollout/analysistemplate.yaml, write kind AnalysisTemplate and metadata.name capa-success-rate, and in spec.metrics[0] fill in name success-rate, interval 30s, failureLimit 3, a successCondition that requires 0.95 or more, and provider.prometheus.address.
Turn one of the canary steps of rollout.yaml into an analysis step so that templates[0].templateName points to capa-success-rate, and set spec.strategy.canary.canaryService to capa-web-canary, stableService to capa-web-stable, and trafficRouting.nginx.stableIngress to capa-web-stable.
In /root/capa-rollout/rollout-bluegreen.yaml, write kind Rollout and metadata.name capa-api, and under spec.strategy.blueGreen put activeService capa-api-active, previewService capa-api-preview, autoPromotionEnabled false, and scaleDownDelaySeconds 60. Do not use canary together in this file.
Create the namespace capa-rollout in the cluster and actually create the Services capa-web-stable and capa-web-canary in it. Both have type ClusterIP, spec.selector.app capa-web, and port 80.
In the same namespace, actually create the Deployment capa-web. replicas is 4, spec.selector.matchLabels.app is capa-web, and the container image is nginx:1.27.
Change the image of capa-web to nginx:1.28 once to create a new revision, and then revert it. The final state must have the image back at nginx:1.27 and a revision number of 3 or higher.

Notes

Use kubectl set image deploy/capa-web nginx=nginx:1.28 -n capa-rollout and kubectl rollout undo deploy/capa-web -n capa-rollout.
You can check revisions with kubectl rollout history deploy/capa-web -n capa-rollout.
Common mistake 1: expecting the revision count to go down after a rollback. A rollback is also recorded as a new revision.
Common mistake 2: using canary and blueGreen together in one Rollout. You can choose only one strategy.

The Rollout skeleton

Create the /root/capa-rollout/ directory and, in rollout.yaml, write apiVersion argoproj.io/v1alpha1, kind Rollout, metadata.name capa-web, metadata.namespace capa-rollout, spec.replicas 4, spec.selector.matchLabels.app capa-web, and the container image nginx:1.27.

A Rollout's spec is almost the same as a Deployment's up to replicas, selector, and template. What differs is what goes under strategy.

Weight and pause steps

Make spec.strategy.canary.steps in the same file five or more, with the first step setWeight: 20, the second pause: {duration: 30s}, the third setWeight: 50, and the last step setWeight: 100.

steps is an array, and each element has one key such as setWeight or pause. If you give pause a duration, it waits for that time; if you do not, it waits indefinitely until a person promotes.

Write the AnalysisTemplate

In /root/capa-rollout/analysistemplate.yaml, write kind AnalysisTemplate and metadata.name capa-success-rate, and in spec.metrics[0] fill in name success-rate, interval 30s, failureLimit 3, a successCondition that requires 0.95 or more, and provider.prometheus.address.

A metrics item needs all of these: how often to measure, what counts as success, how many failures to tolerate, and where to fetch from. If even one of the four is missing, the judgment does not hold.

Connect the analysis step and traffic routing

Turn one of the canary steps of rollout.yaml into an analysis step so that templates[0].templateName points to capa-success-rate, and set spec.strategy.canary.canaryService to capa-web-canary, stableService to capa-web-stable, and trafficRouting.nginx.stableIngress to capa-web-stable.

An analysis step references the AnalysisTemplate name through a templates array. To actually split the traffic ratio, you have to tell the controller the names of the canary Service and the stable Service.

The blue-green strategy

In /root/capa-rollout/rollout-bluegreen.yaml, write kind Rollout and metadata.name capa-api, and under spec.strategy.blueGreen put activeService capa-api-active, previewService capa-api-preview, autoPromotionEnabled false, and scaleDownDelaySeconds 60. Do not use canary together in this file.

Blue-green brings up the new version in full, accesses it only through preview, and then swaps active. There is a field that turns off automatic promotion and a field that decides how long to keep the old version alive.

Create the stable and canary Services

Create the namespace capa-rollout in the cluster and actually create the Services capa-web-stable and capa-web-canary in it. Both have type ClusterIP, spec.selector.app capa-web, and port 80.

From here on it is a real cluster. The selectors of the two Services may be the same. The controller manipulates the selectors later to split the traffic.

Deploy the corresponding Deployment

In the same namespace, actually create the Deployment capa-web. replicas is 4, spec.selector.matchLabels.app is capa-web, and the container image is nginx:1.27.

You must put the same label as the Service selector on the Pod template for the endpoints to be picked up. Match the number of replicas to the Rollout file.

Roll out and roll back

Change the image of capa-web to nginx:1.28 once to create a new revision, and then revert it. The final state must have the image back at nginx:1.27 and a revision number of 3 or higher.

Change the image once to create a new revision, then revert it. Confirm that even after the rollback, the revision number does not go down but goes up.