The Path to etcd — Admission as a Place to Stand

In one sentence

A policy engine is not magic. It is a single hook that makes the API server ask someone else just before it stores a request, and only by knowing where that hook sits can you understand why a policy does not take effect, and why it can halt a cluster.

Why it was needed

RBAC answers "who can do what." If you give a developer permission to create Pods, they can create Pods. But what an organization really wants is the next question. "You may create it, but it must look like this." Images only from the internal registry, resource limits always, no latest tag, labels that follow the standard. RBAC's verbs and resource kinds cannot express these conditions. Permission is per action, "creating," and it has no eyes for the content of the object being created.

So for a long time people filled this gap with people. They wrote the standard in a wiki, pointed it out in PR reviews, and ran a scanner after deployment to find what was wrong. All three methods share the same weakness. They have no enforcement power, or they find the problem after it has already gone in. If there is even one path that does not go through review (running kubectl apply directly, bypassing CI, another team's Helm chart), the rules start to leak.

Admission control solves this problem structurally. It puts the rules not with people but at a chokepoint the API server always passes through. No object enters the cluster without passing that chokepoint, so however many paths there are, the rules need to exist in only one place.

How it works

This is the order that a single kubectl apply goes through inside the API server.

요청 → 인증(Authentication) → 인가(Authorization/RBAC)
     → 뮤테이팅 어드미션(Mutating Admission)
     → 스키마 검증(Object Schema Validation)
     → 밸리데이팅 어드미션(Validating Admission)
     → etcd 저장

There are four facts you must hold on to here.

First, authorization comes first. If you lack permission, the policy is not evaluated at all. A policy is not a replacement for RBAC but a filter that screens a request that has already passed RBAC once more. If you think of the order the other way around, you get wrong designs such as "granting permissions with policy."

Second, mutation comes before validation. This order is not an accident but a design. Only if you check after filling in the defaults can a flow like "the resource limits weren't written, but the policy filled them in, so it passes" hold. The pitfall comes from here at the same time. The object a validation rule sees is already the object after the mutation rules have touched it. If you set up a mutating policy that injects a sidecar and a validating policy that requires limits on every container, and you did not put limits in the injected spec, deployment is blocked and the log prints a container name the user never wrote. It is the typical form of a policy catching its own author.

Third, schema validation sits in between. If a mutation rule produces a field that is not in the schema, it gets caught here. What a policy produces must ultimately also be a Kubernetes object.

Fourth, a webhook is a network call. The API server sends an HTTP request to an external Pod and waits for the answer. So two safeguards are attached to the webhook configuration.

Setting	Meaning	If misused
timeoutSeconds	How long to wait for a response (default 10 seconds, maximum 30 seconds)	If set long, the API server is held up that long during an outage
failurePolicy: Fail	If the webhook cannot respond, the request is rejected	If the policy engine dies, every write in the cluster is blocked
failurePolicy: Ignore	If the webhook cannot respond, the request passes	While the policy engine is down, everything comes in unchecked
namespaceSelector	Makes only certain namespaces go through the webhook	If you do not set it, even kube-system depends on the webhook

failurePolicy is the most important line in this course. Fail protects security, but it means that the policy engine's availability becomes the cluster's availability. If all the admission controller Pods go down, even Pods, ConfigMaps, and the deployment meant to recover that controller are blocked. You fall into a circular dependency. That is why excluding kube-system and the policy engine's own namespace from the webhook targets with namespaceSelector is not a matter of taste but a matter of leaving an escape hatch. It is also why, in real operations, the runbook contains a procedure for bringing a cluster back by deleting the whole webhook configuration.

This is also why a policy that matches every resource with a wildcard is dangerous. The cost of that policy is not the execution time of one policy but a latency tax attached to every request coming into the API server. Narrowing the match scope by kind and namespace is not performance tuning but availability work.

What you see in the field

First, "no policy blocks anything." No matter how hard you stare at the policy YAML, there is no answer. Usually the cause is that the webhook is not registered, or that the match block does not fit the actual request so nothing matches. A policy that matches nothing quietly lets everything through, so it cannot be told apart from a working policy by the human eye. That is why, when writing a policy, you need the habit of trying both a resource that should pass and a resource that should be blocked.

Second, when a webhook failure spreads into a total outage. If three admission controllers are crowded onto the same node and that node drops out, then under failurePolicy: Fail every write in the whole cluster stops. This is why increasing replicas, setting a PodDisruptionBudget, and enforcing node spreading are mandatory steps in adopting policy.

Third, the honest limits of this lab environment. The kwok cluster in this environment runs a real etcd, API server, controller manager, and scheduler, but no policy engine controller is running. So even if you kubectl apply a bad Pod, the cluster does not block it. Background scans do not run either, and resources that generate rules would create do not actually appear. Instead, you can run the policy engine locally with the kyverno CLI and see the same decision logic. You learn this document's webhook order, failurePolicy, and timeouts through reading and quizzes rather than labs, and in the labs you build the ability to write the policies themselves.

What to look for in the next check

In the quiz that follows, you first distinguish the admission order, webhook timeouts and failurePolicy, and the points at which Audit and Enforce apply. After confirming those concepts, in the next module you write a ClusterPolicy that requires mandatory labels, along with pass and reject fixtures.