The policy was applied and nothing got blocked

Goal

Using the ValidatingAdmissionPolicy (CEL) built into Kubernetes, you create a rule that checks Pods, raise it from a warning to a block, put the violating value in the message, and attach exceptions, scope, and parameters yourself.

Why it matters

ValidatingAdmissionPolicy is what made it possible to do validation inside the API server without installing an external policy engine. Unlike a webhook, there is no network round trip and no certificate, and the cluster does not halt because an engine died. In exchange, there is one more thing to learn — that the decision rule and the scope of application are different objects. Thanks to this separation, you can attach the same rule as a warning for some teams and as a block for others, and you can pull only the rule's values out into a ConfigMap and change them without touching the policy. Conversely, if you do not know this separation, you wander for a long time at the point of "I definitely applied the policy but nothing is blocked." In practice, when you first introduce a policy, you always start with Warn to see what gets caught, exclude system components with matchConditions, and then narrow the scope and raise it to Deny. A policy that skips this order stops deployments, gets rolled back, and then nobody turns it on again.

Steps

Work in /root/vap (export KUBECONFIG=/root/.kube/config, kubectl config use-context kwok-lab). In /root/vap/policy.yaml, write a ValidatingAdmissionPolicy require-cpu-limit of admissionregistration.k8s.io/v1. matchConstraints.resourceRules catches CREATE and UPDATE on v1 pods in the core group (""), and the validation requires that every container have resources.limits.cpu. Also create two test Pods — in /root/vap/pod-nocpu.yaml, container web-tier has only a memory limit, and in /root/vap/pod-ok.yaml, container web-full has both cpu and memory limits. After applying the policy, create a namespace team-open that has no labels at all, send pod-nocpu.yaml to it with kubectl create --dry-run=server, and save that output to /root/vap/01-nobinding.txt.
Create a namespace team-warn and attach the label cpu-limit=warn. In /root/vap/binding-warn.yaml, write a ValidatingAdmissionPolicyBinding cpu-limit-warn — policyName is require-cpu-limit, validationActions is ["Warn", "Audit"], and matchResources.namespaceSelector.matchLabels is cpu-limit: warn. After applying, send pod-nocpu.yaml to team-warn with --dry-run=server and save the output to /root/vap/02-warn.txt (including standard error). Even though a warning appears, the Pod must be created.
Create a namespace team-deny and attach the label cpu-limit=deny. In /root/vap/binding-deny.yaml, write a second binding cpu-limit-deny — the same policyName, validationActions of ["Deny"], and a selector of cpu-limit: deny. After applying, send pod-nocpu.yaml to team-deny and save the output to /root/vap/03-deny.txt, and then send pod-ok.yaml too and check that it passes. Do not delete the warning binding (cpu-limit-warn) but leave it as it is — having both scopes alive at the same time is what a real rollout looks like.
Edit /root/vap/policy.yaml and put noCpu in spec.variables — compute the list of names of containers that have no cpu limit. Change the validation to size(variables.noCpu) == 0, and instead of message, use messageExpression to build a message containing those names and the Pod name. And in /root/vap/pod-two.yaml, create a Pod probe-two with two containers — web-nolimit has no resources at all, and cache-ok has both cpu and memory limits. After reapplying the policy, send it to team-deny and save the rejection message to /root/vap/04-message.txt. The message must show only web-nolimit and must not show cache-ok.
Add spec.matchConditions to /root/vap/policy.yaml. There are two conditions — skip-kube-system-sa makes the policy skip if the requester is a service account that starts with system:serviceaccount:kube-system:, and skip-exempt-pods makes it skip if the Pod has the label cpu-limit-exempt: "true". In /root/vap/pod-exempt.yaml, create a Pod probe-exempt with that label (container web-tier, memory limit only). After reapplying the policy, check two things in team-deny and save the output to /root/vap/05-skipped.txt — (1) kubectl --as=system:serviceaccount:kube-system:replicaset-controller create -n team-deny -f pod-nocpu.yaml --dry-run=server, and (2) sending pod-exempt.yaml as usual. Both must be created. pod-nocpu.yaml without the label must still be rejected.
Create the namespaces vap-params and team-images (label registry-check=on). In vap-params, create a ConfigMap allowed-registries and set the value of key allowed to exactly registry.internal/,ghcr.io/labhub/. In /root/vap/policy-registries.yaml, write a second policy allowed-registries — spec.paramKind is apiVersion: v1, kind: ConfigMap, it catches Pods with the same matchConstraints, and it rejects images that start with none of the items in the comma-split list of params.data['allowed']. In /root/vap/binding-registries.yaml, write a binding allowed-registries — validationActions is ["Deny"], paramRef is that ConfigMap (name, namespace, parameterNotFoundAction: Deny), and the selector is registry-check: "on". In /root/vap/pod-img-bad.yaml, create a Pod probe-img-bad whose image is docker.io/nginx:1.27 (container web-img, with cpu and memory limits), send pod-ok.yaml and pod-img-bad.yaml to team-images in turn, and save the two outputs to /root/vap/06-param.txt.
Create /root/vap/scope.sh <네임스페이스> (taking a namespace as its argument). Send /root/vap/pod-nocpu.yaml to that namespace with --dry-run=server, and if it is rejected, print one line DENY and exit with code 3; if only a warning appears and it is created, WARN and 2; and if it is created with no message, OPEN and 0. The output is only that one word on one line. Do not judge by reading the namespace name or labels; judge by the result of the actual request — the grader briefly changes the labels of team-open while calling this script and then restores them. After creating it, run it in turn against team-deny, team-warn, and team-open, and save the results to /root/vap/07-scope.txt, one line each in the form <네임스페이스> <낱말> (namespace, then the word).
Add a second rule to /root/vap/policy.yaml — a variable noMem (the list of names of containers that have no memory limit), and a second validation that checks size(variables.noMem) == 0. This validation's messageExpression must contain the word memory and that container's name, and the first validation's message must contain the word cpu and that container's name. In /root/vap/pod-mixed.yaml, create a Pod probe-mixed — container web-tier has only a memory limit, and cache-tier has only a cpu limit. After reapplying the policy, send this Pod to team-warn and team-deny in turn and save the two outputs to /root/vap/08-two-rules.txt. The Warn side reports both rules and the Deny side reports only one.

Notes

You start with export KUBECONFIG=/root/.kube/config and kubectl config use-context kwok-lab. It is a real kube-apiserver v1.30.4 that kwok runs inside the Pod, and you keep all outputs under /root/vap.
kwok does not actually run Pods. All this lab looks at is the admission stage, so kubectl create -f <파일> --dry-run=server (with a file name in place of the placeholder) is enough — it runs admission as it is and leaves no object.
Right after you change a policy or binding, it is reflected a round trip or two late. If the result looks like the old one, send it again a few seconds later.
Common mistake: applying only the policy and leaving out the binding. A policy is not even evaluated until a binding points to it.
Common mistake: capturing the warning to a file without 2>&1. Warnings come out on standard error.
Common mistake: writing matchConditions as positive. The policy looks at the request when the condition is true, so a condition you want to skip must be written as a negation.
Validating Admission Policy · Common Expression Language in Kubernetes · Admission Controllers Reference

Apply only the policy and nothing happens

Work in /root/vap (export KUBECONFIG=/root/.kube/config, kubectl config use-context kwok-lab). In /root/vap/policy.yaml, write a ValidatingAdmissionPolicy require-cpu-limit of admissionregistration.k8s.io/v1. matchConstraints.resourceRules catches CREATE and UPDATE on v1 pods in the core group (""), and the validation requires that every container have resources.limits.cpu. Also create two test Pods — in /root/vap/pod-nocpu.yaml, container web-tier has only a memory limit, and in /root/vap/pod-ok.yaml, container web-full has both cpu and memory limits. After applying the policy, create a namespace team-open that has no labels at all, send pod-nocpu.yaml to it with kubectl create --dry-run=server, and save that output to /root/vap/01-nobinding.txt.

A policy only defines "how to judge," and "where to apply" is decided by a separate binding object. So if you apply only the policy, the API server does not even evaluate that expression. To check a whole list in CEL, use all(...), and check a field that may be absent first with has(...). kubectl create -f <파일> --dry-run=server (with a file name in place of the placeholder) runs admission as it is but leaves no object. The rejection message and the warning come out just like a real request.

Attach a binding and warnings start to appear

Create a namespace team-warn and attach the label cpu-limit=warn. In /root/vap/binding-warn.yaml, write a ValidatingAdmissionPolicyBinding cpu-limit-warn — policyName is require-cpu-limit, validationActions is ["Warn", "Audit"], and matchResources.namespaceSelector.matchLabels is cpu-limit: warn. After applying, send pod-nocpu.yaml to team-warn with --dry-run=server and save the output to /root/vap/02-warn.txt (including standard error). Even though a warning appears, the Pod must be created.

A binding decides "this policy, where, and how strongly." Warn only returns a warning through the response header and lets the request pass, and Audit leaves an annotation in the audit log. Warnings come out on standard error, not standard output, so you must add 2>&1 for them to be captured in the file too. The namespaceSelector looks at the labels of the namespace object.

Raise the same policy to Deny and the request is blocked

Create a namespace team-deny and attach the label cpu-limit=deny. In /root/vap/binding-deny.yaml, write a second binding cpu-limit-deny — the same policyName, validationActions of ["Deny"], and a selector of cpu-limit: deny. After applying, send pod-nocpu.yaml to team-deny and save the output to /root/vap/03-deny.txt, and then send pod-ok.yaml too and check that it passes. Do not delete the warning binding (cpu-limit-warn) but leave it as it is — having both scopes alive at the same time is what a real rollout looks like.

You can attach several bindings to one policy, and each binding has its own scope and its own strength. That is how "the same rule as a warning for some teams and as a block for others" works without copying the policy. The rejection message also prints which policy and which binding blocked it — a clue for finding the cause when several rules overlap.

The rejection message tells you which container

Edit /root/vap/policy.yaml and put noCpu in spec.variables — compute the list of names of containers that have no cpu limit. Change the validation to size(variables.noCpu) == 0, and instead of message, use messageExpression to build a message containing those names and the Pod name. And in /root/vap/pod-two.yaml, create a Pod probe-two with two containers — web-nolimit has no resources at all, and cache-ok has both cpu and memory limits. After reapplying the policy, send it to team-deny and save the rejection message to /root/vap/04-message.txt. The message must show only web-nolimit and must not show cache-ok.

variables is a device that binds an expression to a name and reuses it as variables.<이름> (variables dot the variable name). Because it is lazily evaluated, it is not computed if the validation does not use it. If you keep only the violators with filter(...) and pull out just the names with map(...), you get a list. You can join a list of strings with join(', '). If messageExpression produces an error, the API server quietly falls back to message — so if the message does not change, suspect the expression.

Create requests the policy does not look at at all

Add spec.matchConditions to /root/vap/policy.yaml. There are two conditions — skip-kube-system-sa makes the policy skip if the requester is a service account that starts with system:serviceaccount:kube-system:, and skip-exempt-pods makes it skip if the Pod has the label cpu-limit-exempt: "true". In /root/vap/pod-exempt.yaml, create a Pod probe-exempt with that label (container web-tier, memory limit only). After reapplying the policy, check two things in team-deny and save the output to /root/vap/05-skipped.txt — (1) kubectl --as=system:serviceaccount:kube-system:replicaset-controller create -n team-deny -f pod-nocpu.yaml --dry-run=server, and (2) sending pod-exempt.yaml as usual. Both must be created. pod-nocpu.yaml without the label must still be rejected.

matchConditions screens once more the requests that matchConstraints picked. The validations are evaluated only when all the conditions are true, so you write a condition you want to skip as a negation. request holds the requester information (request.userInfo.username). If you block even the Pods that controllers create, the cluster can no longer repair itself, so leaving system subjects out is the first safety device of a rollout. You can pretend to be another subject with kubectl --as.

Pull the rule's values out into a ConfigMap and change them without fixing the policy

Create the namespaces vap-params and team-images (label registry-check=on). In vap-params, create a ConfigMap allowed-registries and set the value of key allowed to exactly registry.internal/,ghcr.io/labhub/. In /root/vap/policy-registries.yaml, write a second policy allowed-registries — spec.paramKind is apiVersion: v1, kind: ConfigMap, it catches Pods with the same matchConstraints, and it rejects images that start with none of the items in the comma-split list of params.data['allowed']. In /root/vap/binding-registries.yaml, write a binding allowed-registries — validationActions is ["Deny"], paramRef is that ConfigMap (name, namespace, parameterNotFoundAction: Deny), and the selector is registry-check: "on". In /root/vap/pod-img-bad.yaml, create a Pod probe-img-bad whose image is docker.io/nginx:1.27 (container web-img, with cpu and memory limits), send pod-ok.yaml and pod-img-bad.yaml to team-images in turn, and save the two outputs to /root/vap/06-param.txt.

paramKind decides "what this policy reads as parameters," and the binding's paramRef decides "which object among those." That is why you can attach the same policy to each team with different values. CEL strings have split and startsWith, and lists have exists. What to do when a parameter cannot be found is decided by parameterNotFoundAction — for a security rule, the default should be to block, not to open. Later in this lab, the grader briefly changes the ConfigMap value and restores it. If you wrote the allowlist inside the expression, it will be exposed then.

The same request gets a different answer in each namespace

Create /root/vap/scope.sh <네임스페이스> (taking a namespace as its argument). Send /root/vap/pod-nocpu.yaml to that namespace with --dry-run=server, and if it is rejected, print one line DENY and exit with code 3; if only a warning appears and it is created, WARN and 2; and if it is created with no message, OPEN and 0. The output is only that one word on one line. Do not judge by reading the namespace name or labels; judge by the result of the actual request — the grader briefly changes the labels of team-open while calling this script and then restores them. After creating it, run it in turn against team-deny, team-warn, and team-open, and save the results to /root/vap/07-scope.txt, one line each in the form <네임스페이스> <낱말> (namespace, then the word).

Warnings come out on standard error, so you must merge the outputs to see them. A rejection has a nonzero exit code and a warning has exit code 0 — separate these two first and then look at whether there is a warning. If you set set -e, the script dies first on a rejection. Admission decides the scope by looking at the namespace's labels, so the same manifest gets a different answer depending on where it is.

Put two rules in one policy and Deny and Warn report different things

Add a second rule to /root/vap/policy.yaml — a variable noMem (the list of names of containers that have no memory limit), and a second validation that checks size(variables.noMem) == 0. This validation's messageExpression must contain the word memory and that container's name, and the first validation's message must contain the word cpu and that container's name. In /root/vap/pod-mixed.yaml, create a Pod probe-mixed — container web-tier has only a memory limit, and cache-tier has only a cpu limit. After reapplying the policy, send this Pod to team-warn and team-deny in turn and save the two outputs to /root/vap/08-two-rules.txt. The Warn side reports both rules and the Deny side reports only one.

validations is a list, and each item has its own expression and its own messageExpression. Variables are shared among several validations. Deny ends the request at the first failed item, but Warn returns all the failed ones as warnings — that is why the Warn stage early in a rollout shows "everything that needs fixing" at once. If the two messages cannot be told apart, nobody knows which rule caught it.