TT Lab
Get started
Learn Learning paths Courses

KCSA — Kubernetes Security Associate

Is Today's nginx:latest the Same Image as Yesterday's?

Continue in TT Lab

Goal

Deploy side by side three image notations that point to the same application (pinned by digest, the latest tag, and an internal registry version tag) and compare them, set imagePullPolicy and the pull secret to match each origin, and then use a ValidatingAdmissionPolicy to enforce a rule that admits only "images from an allowed registry or pinned by digest" and observe the rejection.

Why it matters

Of the 4C of cloud native security (Cloud, Cluster, Container, Code), the inner two layers are about what you run. No matter how well you lock down the cluster, if the image a Pod pulls has been swapped by someone, the defense collapses from the inside.

A tag is only a label, so the registry can reattach it to a different image at any time. nginx:latest may be a different image today and tomorrow, and a version tag such as app:1.2.3 is the same if overwriting has not been prevented. By contrast, an @sha256:... digest is a hash of the image contents, so it becomes a different value if even one character changes. So the first step of supply chain security is to sort out "which images are pointed to only by a tag."

imagePullPolicy decides whether a node trusts its cache. If you use IfNotPresent with a tag image, each node may end up running the image from a different point in time. Give the internal registry credentials as a pull secret, and if you attach it to the service account, you do not have to repeat it for every Pod.

Finally, an admission policy is evaluated only at the time of the create request. The Deployments you created without the policy in step 2 remain as they are even after the step 6 policy is turned on. But if a mutable (nginx:latest) Pod is deleted and the ReplicaSet tries to create a new Pod, that create request is rejected — which is why you must inspect existing workloads first before turning on a policy.

Steps

  1. Create the namespace kcsa-supply.
  2. Create the Deployments pinned (digest), mutable (nginx:latest), and internal (registry.example.com/app:1.2.3).
  3. Classify each image as digest or tag and write it to /root/kcsa-supply/classify.txt.
  4. Change tag images to imagePullPolicy: Always and the digest image to IfNotPresent.
  5. Create the docker-registry secret regcred and attach it to the imagePullSecrets of the service account deployer.
  6. Create the policy require-allowed-image and a binding that admit only images from an allowed registry or pinned by digest.
  7. Save the message rejecting the docker.io/library/evil:1 Pod to /root/kcsa-supply/denied-image.txt.
  8. Leave the inspection results in /root/kcsa-supply/report.txt.

Notes

Create a work area to examine origins

Create the namespace kcsa-supply. The policy binding in a later step selects this namespace with the automatically attached kubernetes.io/metadata.name=kcsa-supply label.

Every namespace automatically gets a kubernetes.io/metadata.name label whose value is its own name. If you generate the create with --dry-run=client and apply it, it is safe to run several times.

The same nginx, but all three origin notations differ

Create three Deployments in the namespace kcsa-supply (each with replicas 1, matching the selector and the Pod labels). pinned uses the image nginx@sha256:0000000000000000000000000000000000000000000000000000000000000000, mutable uses nginx:latest, and internal uses registry.example.com/app:1.2.3. The first container image of each Deployment must be exactly this string.

An image reference has the shape [레지스트리/]저장소[:태그][@sha256:다이제스트] (the Korean words stand for registry, repository, tag, and digest). kubectl create deployment makes up a container name from the image string, and for an image containing @sha256: the name may violate the rules and fail. It is safer to create them with a YAML manifest in which you write the container name yourself. The digest must be 64 hexadecimal digits to be syntactically valid.

Sort out which images can change tomorrow

Classify the images of the three Deployments as digest (pinned by digest) or tag (pointed to by a tag) and write them to /root/kcsa-supply/classify.txt one line each in 이름=분류 format (pinned, mutable, internal; the placeholders are the name and the classification).

A tag is a label the registry side can repoint to a different image at any time, and a digest is a hash of the image contents, so it does not change. Not only latest but also a version tag like 1.2.3 is ultimately a tag. Look at the actual image string with kubectl get deploy -o jsonpath and separate them by whether @sha256: is present.

Do not trust tags; make it re-check every time

Change the container imagePullPolicy of the Deployments — Always for mutable and internal, which use tags, and IfNotPresent for pinned, which is pinned by digest.

A tag's target can change, so even if the node cache has an old image with the same name, it must ask the registry again. A digest is a content hash, so using it from the cache as is is safe. The default is Always if the tag is latest or absent and IfNotPresent otherwise, so you must change internal yourself. Fix the corresponding container (matched by name) in spec.template.spec.containers with kubectl patch.

Attach internal registry credentials to the service account

In the namespace kcsa-supply, create a docker-registry type secret regcred (server registry.example.com, user u, password p), create the service account deployer, and put regcred in its imagePullSecrets.

kubectl create secret has a docker-registry subcommand that creates a secret of type kubernetes.io/dockerconfigjson (flags for server, user, and password). If you attach imagePullSecrets to the service account, you do not have to write the secret separately for every Pod that comes up with that SA. After creating the SA, add the field with patch.

Do not admit images from outside the allowed registry

Create a ValidatingAdmissionPolicy require-allowed-image and a binding require-allowed-image-binding. The policy admits a Pod CREATE only if every container image starts with registry.example.com/ or contains @sha256: (message image must come from registry.example.com or be digest-pinned, failurePolicy: Fail), and the binding applies with validationActions ["Deny"] to only the kubernetes.io/metadata.name: kcsa-supply namespace.

CEL strings have startsWith() and contains(). You must include the / at the end of the prefix so that a name like registry.example.com.evil.io/... cannot slip through. Remember that the policy sets what to check, and the binding sets the scope of application and the action (Deny).

A strange image from Docker Hub is blocked at the door

In the namespace kcsa-supply, try to create a Pod with the image docker.io/library/evil:1, and save the entire rejection message the API server returns to /root/kcsa-supply/denied-image.txt.

When it is rejected, kubectl ends with a nonzero exit code and writes the error to standard error. Capture standard error in the file too. Right after the binding, it may take 1–2 seconds to take effect, so if the Pod got created, delete it and try again. Also check that the Deployment Pods from step 2 are still alive — admission applies only to create requests.

Leave a ledger of the image provenance inspection results

Write the inspection results to /root/kcsa-supply/report.txt in four lines — pinned=digest, mutable=tag, latest-pullpolicy=Always, untrusted-registry=denied. The grader compares each line with the actual Deployment spec and the result of trying to create a Pod with an image outside the allowed list.

Carry over the values you confirmed directly in the earlier steps. latest-pullpolicy is the actual imagePullPolicy value of the mutable container (with the case as it is), and untrusted-registry is the result you saw in step 7 (denied or allowed).