Watch What the Security Settings Actually Block

This lab runs on a cluster where security is actually enforced

A real k3s is running inside the VM. Because the kubelet and containerd actually run, a Pod that violates securityContext is really rejected, and a read-only root really blocks writes.

The other labs in the CKS course run on kwok. There, even violating Pods just become Running, so you practice writing configuration, but you don't practice confirming that the configuration is enforced.

It takes about 2 minutes to come up the first time.

Goal

You distinguish when and where a security control blocks. What is blocked at admission and what is blocked at runtime differ in both symptom and the place to fix.

Why it matters

In practice, the place where accidents happen is not "writing the configuration wrong" but "writing all the configuration and then it not being applied." The latter is quiet.

And even for the same "the Pod won't come up," the causes are scattered across several layers.

Where it is blocked	Symptom	Where to fix
Admission (at creation time)	kubectl apply itself is rejected	Namespace labels, policy
kubelet (when creating the container)	The Pod is created, CreateContainerConfigError	securityContext
Runtime (inside the process)	The Pod is Running, only the application fails	Volumes, capabilities

If you can't tell these three apart, every outage means scanning everything from the beginning.

Steps

1. Bring up an image that runs as root with a Pod that has runAsNonRoot: true. Put the rejection reason in /root/cks/nonroot.txt, and also bring up a correct Pod (good-nonroot). The violating Pod's name is bad-root.
2. Give the hardened Pod readOnlyRootFilesystem: true, confirm that writing is really blocked, and put it in /root/cks/readonly.txt. You must also give it a place to write temporary files.
3. On the same Pod, drop capabilities to ALL and give allowPrivilegeEscalation: false, then put the effect in /root/cks/caps.txt.
4. Apply Pod Security Admission at restricted to the locked namespace, and put in /root/cks/psa.txt that a privileged Pod is rejected at creation time.
5. In locked, create the app-sa service account and app-role to allow only reading configmaps. Put the result in /root/cks/rbac.txt.
6. Create app-secret, mount it in a Pod, check what that file looks like inside the Pod, and put it in /root/cks/secret.txt.
7. Write an audit policy in /root/cks/audit-policy.yaml. In the first rule, protect core secrets, configmaps, and serviceaccounts/token with Metadata, with no user, verb, or namespace restrictions. Next, only /healthz, /readyz, /livez and each of their /* subpaths get None, and the last is an unconditional Metadata. omitStages may omit only RequestReceived. Use 3 rules in total, or 4 rules by adding a detailed RBAC write rule before the default. The optional detailed rule records only create, update, patch, delete, and deletecollection on roles, rolebindings, clusterroles, and clusterrolebindings in rbac.authorization.k8s.io at RequestResponse. In /root/cks/audit.txt, explain the level, first match, body protection, and log retention reasons. This step is policy writing and is not proof of applying it to the API.
8. In /root/cks/report.md, write the three lines nonroot_reject_reason=, psa_level=, and rbac_denied=, along with a summary of when and where things are blocked.

Notes

• You look at the violating Pod's state with kubectl get pod bad-root -o jsonpath='{.status.containerStatuses[0].state}'.
• The PSA label is pod-security.kubernetes.io/enforce=restricted. You can also apply audit and warn separately, and in practice you apply warn first to see what gets caught and then raise to enforce.
• To check permissions, use kubectl auth can-i <동사> <자원> --as=system:serviceaccount:<ns>:<sa> -n <ns> (the placeholders are the verb and the resource).
• If you use a read-only root, you must attach an emptyDir to places like /tmp. Otherwise most programs that write temporary files die.
• Common mistake 1: giving only runAsNonRoot: true and not giving runAsUser. If the image runs as root it is rejected, but that fact is only visible by looking at the image, so it is confusing.
• Common mistake 2: applying PSA as enforce right away. Workloads that are already running are all blocked at the next redeployment. Measure with warn first.

This is a 70-minute lab, so extend it with the +time button before it expires (up to 180 minutes). Your work disappears when the session ends.

If it runs as root, it doesn't come up at all

Bring up an image that runs as root with a Pod that has runAsNonRoot: true. Put the rejection reason in /root/cks/nonroot.txt, and also bring up a correct Pod (good-nonroot). The violating Pod's name is bad-root.

runAsNonRoot: true is decided by the kubelet, which looks at the image's USER just before creating the container. The point is that it is the kubelet that blocks it, not admission.

Make the root read-only

Give the hardened Pod readOnlyRootFilesystem: true, confirm that writing is really blocked, and put it in /root/cks/readonly.txt. You must also give it a place to write temporary files.

If you give only readOnlyRootFilesystem: true, programs that write temporary files die. Attach an emptyDir along with it.

Drop all capabilities

On the same Pod, drop capabilities to ALL and give allowPrivilegeEscalation: false, then put the effect in /root/cks/caps.txt.

After drop: ["ALL"], add back with add only what is truly needed. Most applications need nothing.

Block it at creation time

Apply Pod Security Admission at restricted to the locked namespace, and put in /root/cks/psa.txt that a privileged Pod is rejected at creation time.

Attach the label pod-security.kubernetes.io/enforce=restricted to the namespace. This is admission, so kubectl apply itself is rejected.

Give only what is needed

In locked, create the app-sa service account and app-role to allow only reading configmaps. Put the result in /root/cks/rbac.txt.

With kubectl auth can-i --as=system:serviceaccount:<ns>:<sa>, check both the allowed and the denied. Looking only at what is open is half of it.

How a Secret looks inside the Pod

Create app-secret, mount it in a Pod, check what that file looks like inside the Pod, and put it in /root/cks/secret.txt.

After mounting it as a volume, look inside with ls -la. The point is the ..data symlink structure and the storage medium.

Choose what to keep

Write an audit policy in /root/cks/audit-policy.yaml. In the first rule, protect core secrets, configmaps, and serviceaccounts/token with Metadata, with no user, verb, or namespace restrictions. Next, only /healthz, /readyz, /livez and each of their /* subpaths get None, and the last is an unconditional Metadata. omitStages may omit only RequestReceived. Use 3 rules in total, or 4 rules by adding a detailed RBAC write rule before the default. The optional detailed rule records only create, update, patch, delete, and deletecollection on roles, rolebindings, clusterroles, and clusterrolebindings in rbac.authorization.k8s.io at RequestResponse. In /root/cks/audit.txt, explain the level, first match, body protection, and log retention reasons. This step is policy writing and is not proof of applying it to the API.

The protection rule must match first. omitStages is not body removal. Distinguish writing the policy from actually collecting completed events.

When and where it is blocked

In /root/cks/report.md, write the three lines nonroot_reject_reason=, psa_level=, and rbac_denied=, along with a summary of when and where things are blocked.

Along with the three lines nonroot_reject_reason=, psa_level=, and rbac_denied=, summarize the three layers: admission, kubelet, and runtime.