Having Written a Policy and Having It Enforced Are Different

In one line

For Kyverno, blocking is the reason for its existence, but in an environment with no blocking part, nothing is verified other than the fact that you wrote a policy.

Why it has to be real admission

In the environment with only CRDs loaded in the early policy-writing labs, you have to distinguish a successful save from actual policy execution. This module and the webhook outage lab compare normal requests and violating requests on a real VM where the controllers run.

Kyverno is an admission controller. Blocking is its reason for existing, and you were effectively learning in an environment with no blocking part.

The three actions differ in timing

Action	When	What
mutate	Admission, before validate	Modifies the request
validate	Admission	Rejects or lets it pass
generate	After admission, by a separate controller	Creates other objects

Knowing this order lets you combine policies. If you put "insert it if it is missing" (mutate) together with "it must be there" (validate), the user passes without doing anything.

And the fact that generate is outside admission produces two things — it needs separate permissions, and it is not created immediately. If the permissions are missing, the policy looks fine but nothing gets created, and the error is only in the Kyverno log.

The most dangerous thing is a silent failure

Even if the Kyverno Pod is Running, nothing happens if the webhook configuration is absent. The policies are there and there is no error. A state in which you have written all the policies and not one is applied looks normal.

That is why, after writing a policy, you must test directly whether it blocks.

One policy can lock up a cluster

If a system Pod that comes within a policy's target violates the required condition, its recreation can be rejected. That does not mean, however, that every policy blocks the system, that existing Pods are terminated immediately, or that even deleting the policy is always blocked. You have to check the actual request kinds, the target scope, and the recovery path.

The order for deploying policies

A new policy is enforced after collecting the violation status and securing time for teams to fix it.

1. Audit + 보고서        적용 대상의 위반을 수집한다
2. 수정·예외·복구 시험   정상 요청과 위반 요청을 모두 시험한다
3. Enforce + 관찰        위반 요청을 거부하고 영향을 관찰한다

The Kyverno ClusterPolicy in this lab is Audit → Enforce. Warn is not the third value of this field. To announce Audit violations as response warnings, set spec.emitWarning as well. Do not put another policy engine's mode names into this API as they are.

From the violation list, distinguish the items to fix from approved exceptions, and switch over by application scope. Do not judge that the next deployment will also pass just because existing Pods keep running.

# Kyverno — 지금 걸리는 것들
kubectl get policyreport -A -o json | jq -r '
  .items[].results[] | select(.result=="fail") |
  "\(.policy): \(.resources[0].namespace)/\(.resources[0].name)"' | sort | uniq -c

Not locking yourself out

If a policy rejects the recovery requests of system components, it can make an outage bigger. The following is a partial example of an exception you can put in a label rule. It is neither a list to apply unconditionally to every system security policy nor a setting that excludes webhook calls.

spec:
  rules:
    - name: require-nonroot
      match:
        any:
          - resources: {kinds: [Pod]}
      exclude:
        any:
          - resources:
              namespaces: [kube-system, kyverno, gatekeeper-system]

When a call to a registered webhook fails, failurePolicy: Fail rejects that request, and Ignore skips that webhook and continues the rest of the processing. A policy-violation denial that was explicitly given as a normal response is still valid even with Ignore. It is neither that the policy object gets deleted nor that the checks of other policies are ignored as well.

A policy rule's exclude is applied inside the engine. The API server's actual call scope is checked through the rules, namespaceSelector, objectSelector, and so on of the WebhookConfiguration. To exclude your own recovery requests when the engine is down, you have to look at this call stage too. Compare the risk of skipping verification against availability, and test an approved recovery path.

A state with no webhook registration and a state with a registration that does not respond are also different. In the follow-up outage lab, you create the two states separately and compare the request results with the actual registrations.

Mutation comes before validation

The admission order is fixed.

변형 웹훅 → 스키마 검증 → 검증 웹훅 → etcd 저장

That is why, if you put together a "policy that fills in default values" and a "policy that requires that field," it passes automatically. Even if the user does not write it, mutation fills it in and validation checks it.

However, you must not forget that mutation changes things without the user knowing. It is confusing if the result stored, as shown by kubectl get -o yaml differs from what you wrote. A mutation policy should document what it changes and, where possible, leave a trace with an annotation.

What really matters in practice

After deploying a policy, test both normal and violating objects. The controller being Running does not by itself guarantee webhook registration and request handling. Check not only the exit code of the create command but also the rejection message and whether it was stored.

When you widen a policy's scope, test system recovery. Distinguish rule exceptions from excluding webhook calls, and check whether real system workloads satisfy the required conditions. Do not apply outage experiments to the whole cluster in production.

If generate does not work, look together at target matching and the background controller's permissions. A Forbidden in the log is grounds for investigating permissions, but you cannot determine the cause from the mere fact that there is no output.

In the next lab you will confirm these directly on real Kyverno.

Checking against the official documentation

• Kubernetes: webhook call scope and failurePolicy
• Kyverno: Audit, Enforce, emitWarning, and policy settings
• Kyverno: ClusterPolicy support scope and deprecation notice

The pinned version in this lab is Kyverno 1.19.1. Do not apply the API of the latest documentation to the existing VM as it is; check it together with the CRDs of the installed version.