Audit policy: keep the trail, remove secret bodies

Goal

On a real k3s in a personal VM, you carry out synthetic secret exposure → Metadata recovery → missing evidence under None → final recovery.

Why it matters

Having a policy does not by itself produce a safe audit trail. You must check together whether real requests are kept and whether the body is not copied. You practice part of CNPE's audit-trail competency, and this does not mean that it copies official exam questions or covers the whole security domain. You need Kubernetes resources, RBAC, JSON, jq, and systemd basics.

What is prepared

An Ubuntu 24.04 personal VM, k3s v1.36.4+k3s1, the cnpe-audit Namespace and a synthetic canary, and detailed audit records of a synthetic Secret are prepared. The first setup can take several minutes. All relative paths are under /root/cnpe-audit. The full path of the drop-in is /etc/rancher/k3s/config.yaml.d/95-cnpe-audit.yaml. This is an API server policy file, not a resource to kubectl apply.

This is a 75-minute lab, so extend the default 60-minute session with the +Time control. When the session ends, the files and the VM disappear. Keep any report you need separately after reviewing it for exposed values. Use systemctl restart only inside this VM. Do not touch the host cluster or anyone else's session, and do not put in real credentials or personal information.

Steps

1. Check your own VM's API and audit environment — With kubectl --kubeconfig=/etc/rancher/k3s/k3s.yaml get nodes, confirm the cluster that is a single current host. The Namespace cnpe-audit is dedicated to this lab. The environment has detailed synthetic Secret collection turned on. Do not bring in another cluster's kubeconfig or real secrets.
2. Report whether it is exposed, not the values — Read the prepared unsafe-case.json and unsafe-events.jsonl and write unsafe-report.json. requests is the number of ResponseComplete events for that Secret, body_present is whether a requestObject or responseObject key exists, marker_exposed is whether the case's marker appears in plain text or as base64, and control_present is whether a 200 comparison record for case.control_uri exists. safe_to_accept is true only when the four Secret requests exist, there is no body or marker, and the comparison record exists. Do not copy the original secret values into the report.
3. Write a policy that leaves out the body but keeps the trace — In safe-policy.json, write apiVersion audit.k8s.io/v1, kind Policy, and omitStages [RequestReceived]. There are two rules. The first rule is level Metadata, namespaces [cnpe-audit], and resources [{group: empty string, resources: [secrets]}], and the second is an unconditional level Metadata. This lab uses this restricted policy contract and is not a general-purpose policy editor.
4. Connect the policy file to the running API — Write 95-cnpe-audit.yaml in JSON syntax. In the list of the only key kube-apiserver-arg+, put audit-policy-file=/root/cnpe-audit/safe-policy.json, audit-log-path=/root/cnpe-audit/runtime.jsonl, audit-log-mode=blocking, audit-log-maxsize=5, audit-log-maxbackup=1, and audit-log-maxage=1. Restart the k3s inside the personal VM and check that /readyz is ok. Grading also checks the actual audit level of a new synthetic canary read.
5. Actually send create, read, reject, and delete — From the collection command format in the reference below, choose safe as the step argument and run it. The collector performs creation of this run's UUID synthetic Secret, an administrator read, a read by an unauthorized impersonated user, deletion, and a separate /version comparison request. safe-case.json, safe-events.jsonl, and safe-origin.jsonl must be created. The four requests are 201, 200, 403, and 200, and there must be no Metadata body.
6. Build a summary that does not pass empty records — From safe-case.json and safe-events.jsonl, compute the same five fields as in step 2 and save them in safe-report.json. Preserve the original excerpt from the time of collection together with the analysis file. This report is a summary of this collector's restricted scenario and is not a general-purpose security gate that approves an arbitrary audit log.
7. Classify an event hidden with None as a failure — Make none-policy.json the same as the step 3 policy, but change only the first Secret rule to None. Change only the policy path of the drop-in to this file, and after restarting the personal k3s and checking readiness, run collect none. Compute the same five fields in none-report.json. The Secret records must be 0, the comparison record must exist, and safe_to_accept must be false. The case where the whole audit feature is turned off differs from this counterexample.
8. Restore the safe policy and leave new evidence — Return the drop-in to safe-policy.json and restart the personal k3s. Create new requests with collect final and compute final-report.json. Do not copy the old safe report. Grading checks the new requests and the current API's canary read together. Afterward, do not start another lab; you can simply end this personal session.

Reference

The collection command format is python3 /opt/fixtures/cnpe-audit-lab.py collect <단계>. Replace <단계> with one of safe, none, and final specified in the task. collect is a collection command that explicitly creates and deletes a synthetic Secret. check reads the files and the current API and does not change the canary. If collection fails, it first refreshes the request identifier so that previous evidence is not mistaken for a new request. Because the origin excerpt is stored separately, even if the original log rotates, earlier steps can be regraded. This is not protection against tampering by the student's root, nor evidence of central collection, immutable storage, power failure, or protection of all sensitive resources. Do the full grading after you recover to the final safe policy. While None from step 7 is active, it is normal for step 4, which requires the current safe state, to fail.

Official documentation

• https://kubernetes.io/docs/tasks/debug/debug-cluster/audit/
• https://docs.k3s.io/installation/configuration
• https://training.linuxfoundation.org/certification/certified-cloud-native-platform-engineer-cnpe/

Check your own VM's API and audit environment

With kubectl --kubeconfig=/etc/rancher/k3s/k3s.yaml get nodes, confirm the cluster that is a single current host. The Namespace cnpe-audit is dedicated to this lab. The environment has detailed synthetic Secret collection turned on. Do not bring in another cluster's kubeconfig or real secrets.

An application Pod's logs and the API server audit records are different things.

Report whether it is exposed, not the values

Read the prepared unsafe-case.json and unsafe-events.jsonl and write unsafe-report.json. requests is the number of ResponseComplete events for that Secret, body_present is whether a requestObject or responseObject key exists, marker_exposed is whether the case's marker appears in plain text or as base64, and control_present is whether a 200 comparison record for case.control_uri exists. safe_to_accept is true only when the four Secret requests exist, there is no body or marker, and the comparison record exists. Do not copy the original secret values into the report.

That detailed unsafe events exist is different from being safe. Do not save False as a string.

Write a policy that leaves out the body but keeps the trace

In safe-policy.json, write apiVersion audit.k8s.io/v1, kind Policy, and omitStages [RequestReceived]. There are two rules. The first rule is level Metadata, namespaces [cnpe-audit], and resources [{group: empty string, resources: [secrets]}], and the second is an unconditional level Metadata. This lab uses this restricted policy contract and is not a general-purpose policy editor.

The first matching rule decides the collection level. If you change every record to None, you also lose who read it.

Connect the policy file to the running API

Write 95-cnpe-audit.yaml in JSON syntax. In the list of the only key kube-apiserver-arg+, put audit-policy-file=/root/cnpe-audit/safe-policy.json, audit-log-path=/root/cnpe-audit/runtime.jsonl, audit-log-mode=blocking, audit-log-maxsize=5, audit-log-maxbackup=1, and audit-log-maxage=1. Restart the k3s inside the personal VM and check that /readyz is ok. Grading also checks the actual audit level of a new synthetic canary read.

Without the +, you can overwrite the existing list of another drop-in. Do not apply the policy YAML like a Kubernetes resource.

Actually send create, read, reject, and delete

From the collection command format in the reference below, choose safe as the step argument and run it. The collector performs creation of this run's UUID synthetic Secret, an administrator read, a read by an unauthorized impersonated user, deletion, and a separate /version comparison request. safe-case.json, safe-events.jsonl, and safe-origin.jsonl must be created. The four requests are 201, 200, 403, and 200, and there must be no Metadata body.

If you change only the file to the safe policy and do not restart, the actual collection level stays the same, so it fails.

Build a summary that does not pass empty records

From safe-case.json and safe-events.jsonl, compute the same five fields as in step 2 and save them in safe-report.json. Preserve the original excerpt from the time of collection together with the analysis file. This report is a summary of this collector's restricted scenario and is not a general-purpose security gate that approves an arbitrary audit log.

The existence of the four requests comes first. Do not check only the single condition that there is no body.

Classify an event hidden with None as a failure

Make none-policy.json the same as the step 3 policy, but change only the first Secret rule to None. Change only the policy path of the drop-in to this file, and after restarting the personal k3s and checking readiness, run collect none. Compute the same five fields in none-report.json. The Secret records must be 0, the comparison record must exist, and safe_to_accept must be false. The case where the whole audit feature is turned off differs from this counterexample.

A successful application of the None policy is not success of the security audit requirement.

Restore the safe policy and leave new evidence

Return the drop-in to safe-policy.json and restart the personal k3s. Create new requests with collect final and compute final-report.json. Do not copy the old safe report. Grading checks the new requests and the current API's canary read together. Afterward, do not start another lab; you can simply end this personal session.

A log that was safe in the past alone cannot prove the configuration that is running now.