The exit code was 0, so the pipeline stayed green for weeks

Goal

You extract a real OpenTofu plan as JSON and read its structure, then apply policy to that JSON to catch destruction, replacement, and missing tags before they reach the cluster or the cloud. And you build a gate you can trust on top of a tool that finds violations and still exits with 0.

Why it matters

Admission control looks at requests that have already been made. But dangerous changes such as replacing a database or making a bucket public do not pass through the Kubernetes API and go straight to the cloud. Those changes have one thing in common — there is a planning stage before applying, and that plan is a structured document you can extract as JSON. If you apply policy to it, you can block when there is not yet anything to revert. But a plan has values mixed in that are decided only when you apply, so if you write rules without knowing those places, false positives pour in and the gate is turned off within days. And if you leave the decision to the tool's exit code, a single tool that finds violations and still exits with 0 can leave a pipeline quietly green for weeks. So this lab deals as much with "what do you base the decision on" as with how to write the rules.

Steps

In /root/tfpolicy/main.tf, require the local, random, and null providers and declare four resources. The triggers of null_resource.api are owner = "platform" and env = "dev", the triggers of null_resource.worker are owner = "data" and env = "dev", local_file.config writes one line v1 (with a trailing newline) to ${path.module}/out/config.txt, and terraform_data.release has an input of v1. Apply with tofu init and tofu apply -auto-approve, then save a plan with no remaining changes to /root/tfpolicy/base.tfplan and create /root/tfpolicy/base.json with tofu show -json.
Edit main.tf like this. Delete the null_resource.worker block, add random_pet.suffix (length = 2), and add null_resource.cache (triggers are owner = "", env = "dev", and name = random_pet.suffix.id). Change the content of local_file.config to one line v2 and the input of terraform_data.release to v2. Do not apply. Save the plan to /root/tfpolicy/change.tfplan and create /root/tfpolicy/change.json, then in /root/tfpolicy/changes.txt write, for each resource that is not no-op, one line <주소> <create|update|replace|delete> (the address, then the action). If a delete and a create are both in one resource, write it as a single replace line.
Read resource_changes[].change.after_unknown of /root/tfpolicy/change.json and create /root/tfpolicy/unknown.txt. For each resource that has at least one unknown leaf (a place whose value is true), write one line <주소> <모르는 잎 개수> (the address, then the number of unknown leaves). A nested place such as triggers.name also counts as one leaf. Do not write resources with no unknown leaves. Line order is not checked.
In /root/tfpolicy/policies/block.yaml, write a policy with apiVersion: json.kyverno.io/v1alpha1 and kind: ValidatingPolicy. Put in one rule that passes only if no resource has delete in change.actions (both a pure delete and a replacement get caught). Then run KYVERNO_EXPERIMENTAL=true kyverno json scan --payload /root/tfpolicy/change.json --policy /root/tfpolicy/policies/block.yaml and save the entire output and the exit code to /root/tfpolicy/scan-exit.txt. Append the exit code as a final line in the form exit=<코드> (where the placeholder is the code). Also run the same scan on /root/tfpolicy/base.json and confirm with your own eyes that it passes.
Create /root/tfpolicy/gate.sh <계획JSON> (taking the plan JSON as its argument). It runs a scan with policies/block.yaml and parses the report. For each violation line (a line containing FAILED or ERROR:), print it one per line with BLOCK prepended, and print RESULT block=<위반 줄 수> (where the placeholder is the number of violation lines) on the last line (more words may follow). It exits with 0 if there are no violations and a nonzero value if there is even one. If the report has no decision line (PASSED, FAILED, ERROR:) at all, the tool output has changed, so it exits with 2, and it also exits with 2 when the file given as the argument does not exist. The grader runs this script with a clean plan and a violating plan that it made itself.
Add a second rule to /root/tfpolicy/policies/block.yaml. A resource that has change.after.triggers passes only if change.after.triggers.owner exists and is not an empty string. However, a resource whose value is not yet known at the plan stage (change.after_unknown.triggers.owner is true) is not counted as a violation. After adding the rule, rerun ./gate.sh /root/tfpolicy/change.json and check that there are two violation lines. The grader tests this rule with three plans: one where owner is empty, one where owner is a not-yet-known value, and a clean one.
In /root/tfpolicy/policies/warn.yaml, create a second policy file. Put in one rule that passes only if the top-level resource_drift of the plan JSON is empty (a plan with no such key at all must also pass). And edit /root/tfpolicy/gate.sh so that it runs the two policies separately. Violations of block.yaml are printed with BLOCK prepended and make the exit code nonzero, and violations of warn.yaml are printed with WARN prepended but do not change the exit code. Change the last line to RESULT block=<차단 위반 수> warn=<경고 위반 수> (the number of blocking violations and the number of warning violations). If the warning policy's report also has no decision line at all, exit with 2.
Edit /root/tfpolicy/out/config.txt directly, outside Terraform (for example, printf 'hand-edited\n' > /root/tfpolicy/out/config.txt). Then save a plan to /root/tfpolicy/drift.tfplan and create /root/tfpolicy/drift.json — resource_drift appears at the top level. Finally, run ./gate.sh on the three plans base.json, change.json, and drift.json, save the outputs to /root/tfpolicy/reports/base.txt, /root/tfpolicy/reports/change.txt, and /root/tfpolicy/reports/drift.txt respectively, and append EXIT <게이트 종료 코드> (the gate's exit code) to the last line of each file. The grader reruns the gate on the three plans and compares with your reports.

Notes

In the field, many places check plan JSON with Conftest or OPA (Rego), but this Pod has no conftest or opa. Instead, it does the same job with the kyverno json scan of kyverno CLI 1.13.2 — the structure of receiving arbitrary JSON as a payload and judging it with policy is the same, and so is what you learn (the shape of the plan JSON, unknown values, the basis of a decision).
Scan: KYVERNO_EXPERIMENTAL=true kyverno json scan --payload <계획JSON> --policy <정책YAML> (with the plan JSON and the policy YAML in place of the placeholders) — if you leave out the environment variable, the command itself is rejected because it is experimental.
Policy format: apiVersion: json.kyverno.io/v1alpha1 · kind: ValidatingPolicy · spec.rules[].assert.all[].check — the key of a check is a JMESPath expression wrapped in parentheses and the value is the expected value.
To test an expression on its own, use kyverno jp query -i <파일> '<식>' (with a file and an expression in place of the placeholders). A JSON literal in JMESPath is a backtick.
The Pod has OpenTofu 1.9.0 as tofu, and the mirror has only the local, random, null, and tls providers. There is no cloud provider, so the place corresponding to tags is replaced with the triggers map of null_resource.
Common mistake: deciding with if kyverno json scan ...; then. This tool ends with 0 even when there are violations — you confirm it by printing it yourself in step 4.
Common mistake: not distinguishing 0 violations from having failed to read even one line of the report. On the day you upgrade the tool, the gate quietly turns off.
Intermediate outputs the grader does not read: /root/tfpolicy/base.tfplan, change.tfplan, and drift.tfplan (saved plan files) and /root/tfpolicy/out/config.txt.
tofu show · tofu plan · Plan JSON format · kyverno-json · kyverno-json assert · The json command of the kyverno CLI · Conftest · JMESPath specification

Extract the plan as JSON

In /root/tfpolicy/main.tf, require the local, random, and null providers and declare four resources. The triggers of null_resource.api are owner = "platform" and env = "dev", the triggers of null_resource.worker are owner = "data" and env = "dev", local_file.config writes one line v1 (with a trailing newline) to ${path.module}/out/config.txt, and terraform_data.release has an input of v1. Apply with tofu init and tofu apply -auto-approve, then save a plan with no remaining changes to /root/tfpolicy/base.tfplan and create /root/tfpolicy/base.json with tofu show -json.

local, random, and null are downloaded from the mirror inside the Pod, so init works even without the internet. terraform_data is a built-in resource that needs no provider. A plan made right after applying has no-op as the actions of every resource — this is exactly the clean payload to use when testing policy. Save the plan file with -out, and get the JSON by feeding that file to tofu show -json.

Four kinds of action come into one plan at once

Edit main.tf like this. Delete the null_resource.worker block, add random_pet.suffix (length = 2), and add null_resource.cache (triggers are owner = "", env = "dev", and name = random_pet.suffix.id). Change the content of local_file.config to one line v2 and the input of terraform_data.release to v2. Do not apply. Save the plan to /root/tfpolicy/change.tfplan and create /root/tfpolicy/change.json, then in /root/tfpolicy/changes.txt write, for each resource that is not no-op, one line <주소> <create|update|replace|delete> (the address, then the action). If a delete and a create are both in one resource, write it as a single replace line.

resource_changes[] in the result of tofu show -json has address and change.actions. If actions is a two-element array, it is a replacement — counting it twice, as one addition and one deletion, is wrong. Which attributes trigger replacement is decided by the provider, so you can also check it with the # forces replacement marker in the plan output. Line order is not checked in grading.

Count the values the plan does not yet know

Read resource_changes[].change.after_unknown of /root/tfpolicy/change.json and create /root/tfpolicy/unknown.txt. For each resource that has at least one unknown leaf (a place whose value is true), write one line <주소> <모르는 잎 개수> (the address, then the number of unknown leaves). A nested place such as triggers.name also counts as one leaf. Do not write resources with no unknown leaves. Line order is not checked.

after_unknown is an object with the same structure as after, in which only the unknown leaves remain as true and the known leaves are omitted entirely. paths(조건) in jq (where the argument is the condition to test) outputs all the paths of values that satisfy the condition — you can count leaves with [paths(. == true)] | length. The reason this place matters when writing policy is that if you conclude a violation right away because there is no value in after, you get false positives.

Write the rule as data and print the exit code

In /root/tfpolicy/policies/block.yaml, write a policy with apiVersion: json.kyverno.io/v1alpha1 and kind: ValidatingPolicy. Put in one rule that passes only if no resource has delete in change.actions (both a pure delete and a replacement get caught). Then run KYVERNO_EXPERIMENTAL=true kyverno json scan --payload /root/tfpolicy/change.json --policy /root/tfpolicy/policies/block.yaml and save the entire output and the exit code to /root/tfpolicy/scan-exit.txt. Append the exit code as a final line in the form exit=<코드> (where the placeholder is the code). Also run the same scan on /root/tfpolicy/base.json and confirm with your own eyes that it passes.

In kyverno-json's assert.check, the key is a JMESPath expression and the value is the expected value — if you put the expected value 0 on a key wrapped in parentheses, (식) (the parenthesized expression), it becomes "the result of that expression must be 0." To ask whether a particular string is in an array, use contains(배열, '값') (the array and the value to look for). If you want to test an expression separately, use kyverno jp query -i <파일> '<식>' (with a file and an expression in place of the placeholders). To capture the output and exit code together, the form 명령 > 파일 2>&1; echo "exit=$?" >> 파일 (the command, then the output file) is convenient.

Drop the exit code and count the report

Create /root/tfpolicy/gate.sh <계획JSON> (taking the plan JSON as its argument). It runs a scan with policies/block.yaml and parses the report. For each violation line (a line containing FAILED or ERROR:), print it one per line with BLOCK prepended, and print RESULT block=<위반 줄 수> (where the placeholder is the number of violation lines) on the last line (more words may follow). It exits with 0 if there are no violations and a nonzero value if there is even one. If the report has no decision line (PASSED, FAILED, ERROR:) at all, the tool output has changed, so it exits with 2, and it also exits with 2 when the file given as the argument does not exist. The grader runs this script with a clean plan and a violating plan that it made itself.

The key to this step is not using if kyverno json scan ...; then — that tool ends with 0 even when it finds violations. Capture the output in a variable, count with grep -E, and end according to whether the counted value is 0. You must separate the judgment "0 violations" from the judgment "could not read anything" — when the tool is upgraded, the latter appears looking like the former. set -e kills the script first when grep finds nothing, so do not use it.

I required tags and even not-yet-known values got caught

Add a second rule to /root/tfpolicy/policies/block.yaml. A resource that has change.after.triggers passes only if change.after.triggers.owner exists and is not an empty string. However, a resource whose value is not yet known at the plan stage (change.after_unknown.triggers.owner is true) is not counted as a violation. After adding the rule, rerun ./gate.sh /root/tfpolicy/change.json and check that there are two violation lines. The grader tests this rule with three plans: one where owner is empty, one where owner is a not-yet-known value, and a clean one.

In a JMESPath filter, JSON literals are written with backticks — `true`, `null`. Reading a key that does not exist gives null, so you must ask both "the key does not exist" and "it is an empty string." And if you do not look at after_unknown, a module that computes the value from a data source shows up wholesale as a violation, and the gate is turned off within days. To test a single rule as a whole, kyverno jp query -i <계획JSON> '<식>' (with the plan JSON and an expression in place of the placeholders) is the fastest.

Some rules block and some rules only notify

In /root/tfpolicy/policies/warn.yaml, create a second policy file. Put in one rule that passes only if the top-level resource_drift of the plan JSON is empty (a plan with no such key at all must also pass). And edit /root/tfpolicy/gate.sh so that it runs the two policies separately. Violations of block.yaml are printed with BLOCK prepended and make the exit code nonzero, and violations of warn.yaml are printed with WARN prepended but do not change the exit code. Change the last line to RESULT block=<차단 위반 수> warn=<경고 위반 수> (the number of blocking violations and the number of warning violations). If the warning policy's report also has no decision line at all, exit with 2.

resource_drift is the place where the plan tells you what someone changed by hand outside the code. Calling length() on a plan with no such key at all produces an error, so it is safer to give a default, as in resource_drift || []``. On the gate side, run the scan twice and hold the counted values separately, and decide the exit code by looking only at the blocking number. This is also the order for turning on a new rule in the field — count with warnings for a few days, and after what gets caught is cleaned up, move it to blocking.

One hand-edited file is caught as drift

Edit /root/tfpolicy/out/config.txt directly, outside Terraform (for example, printf 'hand-edited\n' > /root/tfpolicy/out/config.txt). Then save a plan to /root/tfpolicy/drift.tfplan and create /root/tfpolicy/drift.json — resource_drift appears at the top level. Finally, run ./gate.sh on the three plans base.json, change.json, and drift.json, save the outputs to /root/tfpolicy/reports/base.txt, /root/tfpolicy/reports/change.txt, and /root/tfpolicy/reports/drift.txt respectively, and append EXIT <게이트 종료 코드> (the gate's exit code) to the last line of each file. The grader reruns the gate on the three plans and compares with your reports.

Drift is the state file's remembered value and the real value going out of step, and it shows up in the refresh done when building the plan. When making the reports, you must take the gate's exit code with $? right after the redirection — if even one other command comes in between, that command's code is caught. The conclusion of this lab is that the RESULT lines of the three reports come out different from one another: clean, blocking, and warning split apart at one gate.