Pod Security Admission and securityContext

Goal

You learn how to set a security floor for workloads with nothing but namespace labels, and you complete by hand a Pod spec that actually passes the restricted profile.

Why it matters

PodSecurityPolicy was removed in 1.25. To use it, you had to give the ServiceAccount the "permission to use this PSP," and that structure itself was a privilege escalation path, and it was hard to predict which of several PSPs would apply. Its replacement, PSA, takes just a few lines of namespace labels, and since the three modes enforce, audit, and warn can be turned on separately, gradual adoption is possible. If you apply enforce straight to an existing cluster, workloads get blocked in bulk, so turning on warn and audit first and observing is the standard procedure.

The four things the restricted profile requires (runAsNonRoot, allowPrivilegeEscalation false, capabilities drop ALL, and seccompProfile) are the combination that comes up most often on the CKS. Rather than memorizing, removing them one at a time yourself and seeing what errors appear lasts longer.

The API server in this environment has the PodSecurity admission plugin turned on, so violating Pods are actually rejected.

Steps

Create the namespace cks-psa and attach the label pod-security.kubernetes.io/enforce=baseline.
Create the namespace cks-psa-strict and attach the enforce, audit, and warn labels all as restricted, and the enforce-version, audit-version, and warn-version labels all as latest.
Create a Pod hardened (container name app, image nginx:1.27-alpine) in cks-psa-strict. It must pass restricted, so it needs runAsNonRoot: true and seccompProfile.type: RuntimeDefault at the Pod level, and allowPrivilegeEscalation: false and capabilities.drop: [ALL] at the container level.
Try to apply a Pod bad-pod with a privileged: true container in cks-psa-strict. Save its output (including standard error) to /root/cks-securitycontext/denied.txt. The Pod must not be created.
Create a Pod nonroot-app (container name app) in cks-psa. In the Pod-level securityContext, put runAsNonRoot: true, runAsUser: 10001, runAsGroup: 10001, and fsGroup: 20001.
Create a Pod dropped (container name app) in cks-psa-strict. The Pod-level seccompProfile.type is RuntimeDefault, runAsNonRoot is true, and at the container level put allowPrivilegeEscalation: false, capabilities.drop: [ALL], and readOnlyRootFilesystem: true.
Create a RuntimeClass cks-sandbox. The handler is runsc, and overhead.podFixed is memory 160Mi and CPU 250m.
Create a Deployment payments in cks-psa-strict. Replicas 2, the selector and Pod label are app=payments, the Pod spec's runtimeClassName is cks-sandbox, and the Pod template meets all four of the same conditions as in step 3 so that it passes restricted.

Notes

kubectl label ns cks-psa-strict pod-security.kubernetes.io/enforce=restricted pod-security.kubernetes.io/enforce-version=latest ...
Example run for step 4: kubectl apply -f bad-pod.yaml 2>&1 | tee /root/cks-securitycontext/denied.txt
kubectl explain pod.spec.securityContext and kubectl explain pod.spec.containers.securityContext show different lists of fields.
Common mistake 1: putting capabilities at the Pod level. It is container-level only.
Common mistake 2: if you leave out the -version labels, the profile definition can change at a cluster upgrade and a Pod that was fine can be rejected.

Apply the baseline profile

Create the namespace cks-psa and attach the label pod-security.kubernetes.io/enforce=baseline.

PSA works through namespace labels. The label keys start with pod-security.kubernetes.io/.

Apply all three modes and the version labels

Create the namespace cks-psa-strict and attach the enforce, audit, and warn labels all as restricted, and the enforce-version, audit-version, and warn-version labels all as latest.

Each of enforce, audit, and warn has its own corresponding -version label. If you pin the version, a cluster upgrade doesn't turn into a policy tightening.

A Pod that passes restricted

Create a Pod hardened (container name app, image nginx:1.27-alpine) in cks-psa-strict. It must pass restricted, so it needs runAsNonRoot: true and seccompProfile.type: RuntimeDefault at the Pod level, and allowPrivilegeEscalation: false and capabilities.drop: [ALL] at the container level.

restricted requires four things. If even one is missing, it is rejected, and the error message tells you which field was caught.

One more thing to know. runAsNonRoot: true does not set a UID. It only says "don't run as root," so if the image doesn't specify a UID, on a real node the kubelet can't start the Pod and raises CreateContainerConfigError. On a real cluster, set runAsUser as well or use an image built to run as non-root. This lab environment doesn't actually run the Pod, so that difference doesn't show up.

Confirm that a violating Pod is rejected

Try to apply a Pod bad-pod with a privileged: true container in cks-psa-strict. Save its output (including standard error) to /root/cks-securitycontext/denied.txt. The Pod must not be created.

You must save both the standard output and standard error of apply to the file. Use 2>&1 | tee. If the Pod was created, the policy didn't catch it.

Specify the run-as user and group

Create a Pod nonroot-app (container name app) in cks-psa. In the Pod-level securityContext, put runAsNonRoot: true, runAsUser: 10001, runAsGroup: 10001, and fsGroup: 20001.

runAsUser/runAsGroup/fsGroup go in the Pod-level securityContext, and runAsNonRoot can go on either side. fsGroup changes the group ownership of volume files.

Combining seccomp and capabilities

Create a Pod dropped (container name app) in cks-psa-strict. The Pod-level seccompProfile.type is RuntimeDefault, runAsNonRoot is true, and at the container level put allowPrivilegeEscalation: false, capabilities.drop: [ALL], and readOnlyRootFilesystem: true.

Since it is a restricted namespace, this Pod too must meet all four required conditions to be created.

The RuntimeClass object

Create a RuntimeClass cks-sandbox. The handler is runsc, and overhead.podFixed is memory 160Mi and CPU 250m.

A RuntimeClass is a cluster-scoped resource. The handler value must match, character for character, the runtime name in the node's container runtime configuration.

Putting it together: a deployment with a sandbox runtime

Create a Deployment payments in cks-psa-strict. Replicas 2, the selector and Pod label are app=payments, the Pod spec's runtimeClassName is cks-sandbox, and the Pod template meets all four of the same conditions as in step 3 so that it passes restricted.

A Deployment's Pod template also goes through the PSA check. Write the RuntimeClass name you created earlier in the Pod spec's runtimeClassName.