TT Lab
Get started
Learn Learning paths Courses

KCA — Kyverno Certified Associate

Four Anchors, and Rolling Out Gradually Without Splitting the Policy

Continue in TT Lab

In one line

A validate rule has three syntaxes: pattern, which draws the object's shape as it is, deny.conditions, which rejects with a conditional expression, and CEL expressions. On top of this comes failureAction, which sets the enforcement level per rule, making it possible to layer only a new rule in observation mode without splitting the policy in two.

Why this was needed

The starting point of Kyverno is not to make people learn a new policy language. OPA/Gatekeeper requires learning Rego and pairing two objects, ConstraintTemplate and Constraint. Kyverno draws the object you want to check as it is in YAML and puts an operator in the value position. If a Pod's containers must have a memory limit, you write it in the shape of the Pod spec as it is and put, in the value position, an operator meaning "not empty."

But drawing the shape as it is has limits. Relationships such as "if this field is present, that condition must also be checked" or "this key must not exist at all" cannot be expressed with values. That is why the anchors attached to key names came about.

How it works

The value-position operators are as follows. ?* is a non-empty value, * is any value including null, X|Y is one of the two, !X is a value that is not X, and numeric comparison is possible as in >=256Mi.

There are four kinds of key-position anchors, and this is where the most incidents happen.

Anchor Name Meaning
() Conditional Check the rest only when this key matches this value
=() Equality If this key exists, its value must satisfy the condition
^() Existence At least one element in the array must satisfy the condition
X() Negation This key must not exist

The most common misuse is the negation anchor. Many people read X(privileged) as "the privileged value must be different," but the exact meaning is "the key privileged itself must not exist." Even if the value is explicitly false, it is caught if the key is present. If you want to compare values, you must use a value operator or a deny condition, not an anchor.

foreach checks each element of a collection. There is one trap here too: foreach.list takes a JMESPath expression itself, so you must not wrap it in curly braces. And the idiom used when you must also look at initContainers is request.object.spec.[initContainers, containers][]. Policies that leave out initContainers are actually very common, and it is not attackers but ordinary teams that simply use init containers who end up bypassing the policy.

deny.conditions rejects when the condition is true. You group conditions with all and any, and each condition has three slots: key, operator, and value. Most things that are awkward to express with pattern end up here. CEL is an expression language that became standard in Kubernetes after 1.25, and it has also come into Kyverno as validate.cel. However, it is a useful option when you already operate Kyverno, not a reason to adopt Kyverno. If all you need is to check a single field, Kubernetes' built-in ValidatingAdmissionPolicy is enough, and operating one fewer component means one fewer upgrade target and one fewer thing to worry about regarding webhook certificate renewal.

Last is the enforcement level. In the past, spec.validationFailureAction decided the enforcement level of the whole policy. So if, within one policy, you wanted to block some rule because you were sure of it and only observe another rule, you had to split the policy in two, and as the match block was duplicated the number of things to manage grew. Now it has moved down to spec.rules[*].validate[*].failureAction, and the values are Enforce and Audit. If you do not specify it, the default is Audit. The value of this change is that a flow in which you layer a new rule onto an existing policy as Audit, watch the reports for a few days, and then raise only that rule to Enforce became possible without splitting the policy. The fields that moved together are webhookTimeoutSeconds and failurePolicy, which move to webhookConfiguration.timeoutSeconds and webhookConfiguration.failurePolicy respectively and are marked deprecated from 1.13.

What it looks like in the field

The author's experience with image scanning meshes exactly with this topic. When a certain image was scanned for the first time, 1,247 findings came out, of which 9 were Critical. If you paste this report as it is into the team channel, the result is one of two things. Everyone ignores it, or nobody can touch it and the release gets blocked. Adding one option that keeps only findings with a fix available brought it to 4, and when compared against the list of confirmed real exploitation, the one that had to be handled today was 1.

Policies are just the same. If you apply Enforce across the board from the start, deployments are blocked wholesale, and in the end it is not the policy but the person who made the policy who becomes the bottleneck. On the other hand, if you leave everything as Audit, nobody looks at the reports. The answer is per-rule failureAction. If you raise only a few certain rules to Enforce and observe the rest with Audit, only the items that can be acted on now remain in front of people. It is the same principle as what was learned from the scan report.

What you will do in the next lab

You write a ClusterPolicy in /root/kca-validate/, filling in match, pattern, deny, and exclude one piece at a time, and give each rule a different failureAction. Then you put the same rule into a real cluster as Kubernetes' built-in ValidatingAdmissionPolicy and Binding, and compare with your own eyes how far you can get without a policy engine.