TT Lab
Get started
Learn Learning paths Courses

CKS — Kubernetes Security Specialist

Why Auditing Starts With the Config File and Policy With Default Deny

Continue in TT Lab

In one line

The reason the CIS benchmark starts by scanning the text files in /etc/kubernetes/manifests, and the reason a network policy must start from "default deny," are the same. In both cases the defaults are open in the dangerous direction.

Why this was needed

Kubernetes chose developer-friendly defaults. A Pod can talk to any Pod, the API server accepts anonymous requests as a user called system:anonymous, and the kubelet could leave open a read-only port with no authentication. These defaults share the property that "if you don't configure it, it is open." So the first question of a security review is not "what did we block" but "what did we not block."

That is why the CIS benchmark and kube-bench start with auditing configuration files. The behavior of most control plane components is determined by the command array of a single static Pod manifest. The mere absence of --anonymous-auth=false in that array means that anyone who can reach the cluster can knock on the API without authentication. Reading one line of text is faster than observing a running process, and above all it is left as evidence.

How it works

A NetworkPolicy is an allowlist. If there is no policy at all for a Pod, that Pod is in a fully allowed state. The moment even one policy selects that Pod, the directions listed in that policy's policyTypes flip to "allow only what is stated." So the practical order is always this.

Order Policy Effect
1 podSelector: {} + policyTypes: [Ingress, Egress], no rules Blocks the entire namespace
2 Open only DNS egress (UDP/TCP 53 to kube-dns) Name resolution restored
3 Open ingress only for the needed Pod pairs Minimal connectivity
4 Block node metadata with the except of ipBlock Cuts off the credential theft path

Forgetting step 2 is the most common accident. When you apply default deny, DNS lookups are egress too and get blocked along with everything else, and the application dies not with "connection refused" but with "name not found." That is why it takes time to realize the cause is the network policy.

ipBlock.except is a point that comes up repeatedly on the CKS. The cloud's instance metadata address 169.254.169.254 can return the node's IAM credentials without authentication. If a single Pod is breached, the node's cloud permissions are handed over wholesale. Even if you open egress to 0.0.0.0/0, carving out just this one address with except is the standard approach.

What it looks like in the field

The author's 7-node homelab runs Cilium 1.20.1 in eBPF mode and doesn't install kube-proxy at all. One thing was confirmed here. With the same IP and the same port 80, GET returned 200 and POST returned 403. No sidecar was injected and the application code was unchanged. This is impossible with the basic NetworkPolicy that sees only L3/L4, and it means that the choice of CNI is the expressiveness of security. Conversely, on a CNI that doesn't implement NetworkPolicy at all, like a default Flannel setup, no matter how carefully you write policy objects, nothing gets blocked. The fact that an object was created and the fact that traffic was blocked are separate.

A more painful accident in the same homelab was on the configuration file side. DHCP changed the control plane IP from 10.0.0.111 to 10.0.0.120, and the new address was not in the apiserver certificate's SAN. etcd and kube-apiserver tried to bind to an address that no longer existed and fell into CrashLoopBackOff, but kube-scheduler and controller-manager bind to 127.0.0.1, so their processes were perfectly alive. This "half alive" state makes diagnosis the hardest. It was an incident that taught, through experience, that the value in one file decides the life or death of the whole cluster.

Where your hands actually go in the exam

The cluster setup domain of the CKS comes up in the form of "fix a configuration file and bring the component back." There are a few things your hands have to remember.

If you edit a static Pod manifest, the kubelet restarts it on its own. It takes effect the moment you change a file under /etc/kubernetes/manifests/. If you get the syntax wrong, that component won't come up at all, so make a copy before editing.

cp /etc/kubernetes/manifests/kube-apiserver.yaml /tmp/
# 고친 뒤
crictl ps | grep apiserver          # 새 컨테이너가 떴는지
journalctl -u kubelet -f            # 안 뜨면 이유가 여기 남는다

If you break the API server, kubectl itself stops working, so verify with crictl and journalctl. If you don't know this, you get stuck in the exam room.

Memorize a few frequently tested flags.

Purpose Flag
Block anonymous access --anonymous-auth=false
Audit log --audit-policy-file, --audit-log-path, --audit-log-maxage
etcd encryption --encryption-provider-config
Admission control --enable-admission-plugins=NodeRestriction,...

An audit policy requires mounting the file as well. If you add only the flag and don't add volumes/ volumeMounts, the API server can't find the file and doesn't come up. This is the most common mistake with static Pods.

etcd encryption only takes effect after you turn it on and rewrite the existing data.

kubectl get secrets -A -o json | kubectl replace -f -

Look at the kubelet side too. authentication.anonymous and authorization.mode in /var/lib/kubelet/config.yaml come up as questions. If you change these, you must restart the kubelet yourself (systemctl restart kubelet).

After changing something, always run one verification line. The answer includes the command that shows whether what the question asked for was actually applied.

What you will do in the next lab

In the cks-net namespace, you apply a default deny policy, layer on an egress rule that opens only DNS as an exception, and carve out the path to node metadata with ipBlock.except. You create a TLS Secret for Ingress of type kubernetes.io/tls, and finally you write a kube-apiserver manifest reflecting the CIS recommendations yourself, checking by hand which flags must be present and which must be absent.