A Signature Proves Where It Came From, Not That It Is Safe

In one line

An image signature does not mean the image is safe. It means the image came out of a particular pipeline. An image full of vulnerabilities can be signed too, and that is not a failure of signing but its definition. A verifyImages rule is the mechanism that enforces this proof of origin in admission.

Why this was needed

That an image exists in a registry guarantees nothing. Someone may have built it on a laptop and pushed it by hand, or a tag may have been overwritten later with different contents. No matter how tightly you apply scan gates to the CI pipeline, if an image pushed around that pipeline can come up in the cluster, the gate is closer to a recommendation.

Signing plugs this hole. You sign images with a key that only the pipeline knows (or the pipeline's OIDC identity), and the cluster accepts only images carrying that signature. Then the scan gate is actually enforced. This is the point where scan results turn into enforcement.

How it works

A verifyImages rule pulls all image references out of the Pod spec, picks only those that match the imageReferences patterns, and asks the registry for signatures. The fact that is easy to miss here is that this lookup is a network call that leaves the cluster. From the moment you turn the policy on, the registry is not only the path for pulling images but also the path that decides whether a Pod can be created.

The verifiers are expressed with attestors. There are three methods: a static public key (keys.publicKeys), KMS, and keyless. Keyless signs with a short-lived certificate issued by Fulcio and leaves a record in the Rekor transparency log, and here subject and issuer are the heart of it. The issuer is which OIDC provider, and the subject is who within it. With GitHub Actions, the subject includes the workflow file path and even the reference tag, so if you rename the release workflow file or change the tag rules, everything is blocked from that moment. This precision is the purpose, but it is also an operational burden.

attestors.count is a trap you must know. If you do not specify it, every entry in entries must pass. If you added one key and suddenly everything fails, this is nine times out of ten the cause. If you put in both the old key and the new key and set count to 1, an image signed with either passes, so you can get through the rotation period without downtime, and if you leave count out, only images signed with both keys pass, which gives exactly the opposite result.

An attestation is a statement layered on top of a signature. SLSA provenance carries "which builder this image came from and from which source," and a CycloneDX or SPDX SBOM carries "which components are inside." You can check that content with conditions, so for example you can write a policy that requires a particular library version in the SBOM not to equal a vulnerable version. If you check only the signature, you have not checked who built it, so it is better to also apply a provenance condition.

Let us also sort out four flags. required enforces that all matched images went through verification, verifyDigest enforces the use of digests themselves, skipImageReferences is a list of patterns to take out of matching, and mutateDigest rewrites tags into digests. The last one has a large effect on pipelines. When it is on, what remains in the Pod spec is not a tag but a fixed reference starting with @sha256:, so even if you overwrite the same tag in the registry, even Pods that newly come up through scale-out use the image that was fixed at first. This is the intended immutability, but a pipeline that used to push a tag and delete Pods to get the new image quietly does nothing from this point on.

The cache gets involved in performance. Image verification results are kept in a TTL cache, and this is an installation-level setting, not a policy field. The defaults are enabled true, a maximum of 1000 keys, and a TTL of 60 minutes. In a rollout of twenty replicas, the registry round-trip cost is actually paid only the first time, so the second measurement comes out much faster than the first and only the first deployment after the TTL expires is conspicuously slow. You have to turn the cache off and measure admission latency to get a value close to the worst case you would see when the registry is down.

What it looks like in the field

The author's homelab registry is Harbor, running at 10.0.0.202, which it received from a MetalLB pool. A problem you always meet the moment you use an in-house registry is that there can be two credential paths. A Pod pulling images fine and Kyverno looking up signatures are separate paths. The Pod uses its own imagePullSecrets, and Kyverno uses its own credentials. You give Kyverno its credentials through the --imagePullSecrets argument of the Kyverno deployment or the policy's imageRegistryCredentials. When the author ran into this, the two were completely split. According to the official Kyverno documentation, from 1.18 on Kyverno automatically uses the spec.imagePullSecrets of the Pod under evaluation as registry credentials. So if you run version 1.18 or later, you do not need to list anything separately in the policy when the Pod has a pull secret, and on an earlier version you must supply one of the two above yourself. Either way, if Kyverno cannot read the private registry, the signature looks absent even when it is perfectly there.

The scanning-side experience converges on the same conclusion. One image had 1,247 vulnerabilities in total, 9 of them Critical, and keeping only those with a fix available left 4, and comparing against the list of confirmed real exploitation left 1. And what eliminated those 1,247 was not individual patches but replacing the base image. The application actually linked eight shared libraries, but the image contained 432 packages. After moving to distroless, OS package vulnerabilities went to 0 and what remained were only two npm dependencies of the application. Signing and scanning connect like this. Making the image thin through scanning, signing what passes, and having the cluster accept only what is signed form a set.

One more thing. If you copy only the image to a mirror registry and do not move the signature artifacts along with it, this policy fails for everything without exception. This is what happens most often when bringing images into an air-gapped network, and you must not turn on Enforce until you specify the signature location separately with repository or fix the mirroring pipeline.

What you will do in the next lab

In /root/kca-verify/, you write a verifyImages rule that uses both a static key and keyless, SLSA provenance and SBOM attestation rules, and a validate rule for allowed registries. Then you put onto a real cluster a registry credential Secret, the ServiceAccount that carries it, and a Deployment pinned by digest.