The firewall that broke DNS and the allow rule that widened access

Goal

On real Cilium, you distinguish policy direction, namespace selectors, DNS dependency, the union of allows, and explicit deny.

Why it matters

If you restart the server first when a timeout occurs after a security change, you can erase the cause. Compare the IP control with the name request to find the dependency, and check normal requests and denied requests together. You verify the actual scope of access, not the number of policies. The comparison targets are kept to the end, so the full grading also looks at the current state again.

Use only the personal k3s inside the VM. The new environment uses Cilium 1.20.1 and k3s 1.35.8+k3s1, and it may take several minutes to start. Step preparation alone does not resolve the current step. Until policy propagation finishes, observation results may temporarily differ, so grade again after observing. When the session ends, your working files disappear, so export them first if you need them.

Steps

1. Use the tool's inventory command to save the namespace, name, uid, ip, and identity of the 11 Pods to /root/cca-policy/inventory.json. The role labels of cca-blue/client and cca-red/client are the same, but the namespace labels and identities differ. Read and compare the actual values in the output. If a Pod was recreated, the record must also be updated.
2. In /root/cca-policy/native.yaml, write and apply the native-only NetworkPolicy in cca-blue. podSelector.matchLabels is target=native, policyTypes is [Ingress], and the single from of the single ingress has only podSelector.matchLabels.role=client. native must return 200 only for the blue client, and the blue stranger and the red client must time out. The baseline without a policy must return 200 for all three.
3. In /root/cca-policy/ingress.yaml, write and apply the blue-ingress CiliumNetworkPolicy in cca-blue. endpointSelector.matchLabels.target=ingress, and the single fromEndpoints of the single ingress has only matchLabels.role=client. Only the blue client must get 200, and the stranger and the red client must time out. Do not delete the native policy.
4. In /root/cca-policy/dns-blocked.yaml, write and apply the dns-blocked CNP in cca-blue. endpointSelector.matchLabels.client-mode=blocked. In a single egress, put only toEndpoints.matchLabels={role: api, target: egress} and toPorts.ports=[{port: "8080", protocol: TCP}]. Do not include a DNS allow. The dns-blocked client's request to the egress server by IP must return 200, and the request by Service name must time out.
5. In /root/cca-policy/dns-open.yaml, write and apply the dns-open CNP in cca-blue. It selects client-mode=open. The first egress entry is the API target and TCP 8080 from the previous step as they are. The toEndpoints.matchLabels of the second entry are k8s:io.kubernetes.pod.namespace=kube-system and k8s:k8s-app=kube-dns, and toPorts.ports has two entries, UDP and TCP, of the string 53. Both the IP and the name access of dns-open must return 200. Leave the restriction of dns-blocked as it is.
6. Preserve the initial blue-union CNP. In /root/cca-policy/union.yaml, write and apply the red-union NetworkPolicy in cca-blue. Select target=union and set policyTypes=[Ingress]. In the single from of the single ingress, put namespaceSelector.matchLabels.team=cca-red and podSelector.matchLabels.role=client together. The blue client and the red client must get 200, and the stranger must time out.
7. Preserve the initial blue-deny CNP and red-deny NetworkPolicy. In /root/cca-policy/deny.yaml, write and apply the deny-red CNP in cca-blue. Select target=deny, and in the single fromEndpoints of the single ingressDeny, put k8s:io.kubernetes.pod.namespace=cca-red and k8s:role=client together in matchLabels. The blue client must get 200, and the stranger and the red client must time out.
8. Save the result of the tool's report command to /root/cca-policy/report.json. The three requests the red client sent to baseline, union, and deny must be 200, 200, and timeout respectively. From the output, compare the current Pod UID/IP/identity, the policy UID/spec hash, the request source port, and the Hubble FORWARDED/DROPPED flows on the same node and time. If you changed a policy or a Pod, create the report again. In the final full grading, all the earlier steps must still be alive.

Notes

• Observation tool: python3 /opt/fixtures/cca-policy/runtime.py observe case-name. The case names are baseline, native, ingress, dns-blocked, dns-open, union, and deny.
• Identity: kubectl get cep -A. Policies: kubectl get cnp,netpol -A -o yaml.
• Hubble: hubble observe --server 127.0.0.1:4245 --namespace cca-blue --last 50.
• You can save the task files as JSON or as a single YAML object. Check not only the values but also how the list items are grouped.
• HTTP 000 is not a server response code. When no response was received, also check the actual exit code and the flows.

Record the real Pods and security identities of the two teams

Use the tool's inventory command to save the namespace, name, uid, ip, and identity of the 11 Pods to /root/cca-policy/inventory.json. The role labels of cca-blue/client and cca-red/client are the same, but the namespace labels and identities differ. Read and compare the actual values in the output. If a Pod was recreated, the record must also be updated.

Having the same name is different from being the same endpoint. Read the UID and the set of security labels together.

The standard NetworkPolicy also selects labels

In /root/cca-policy/native.yaml, write and apply the native-only NetworkPolicy in cca-blue. podSelector.matchLabels is target=native, policyTypes is [Ingress], and the single from of the single ingress has only podSelector.matchLabels.role=client. native must return 200 only for the blue client, and the blue stranger and the red client must time out. The baseline without a policy must return 200 for all three.

Do not read the policy's target selector and the incoming-peer selector the wrong way around. Check the scope in which the peer has no namespaceSelector.

Check the boundary between the two teams in a CNP too

In /root/cca-policy/ingress.yaml, write and apply the blue-ingress CiliumNetworkPolicy in cca-blue. endpointSelector.matchLabels.target=ingress, and the single fromEndpoints of the single ingress has only matchLabels.role=client. Only the blue client must get 200, and the stranger and the red client must time out. Do not delete the native policy.

The name CNP alone does not automatically connect all namespaces. Leave the other team with the same role as a control request too.

Reproduce the outage where the API is alive and only name resolution is blocked

In /root/cca-policy/dns-blocked.yaml, write and apply the dns-blocked CNP in cca-blue. endpointSelector.matchLabels.client-mode=blocked. In a single egress, put only toEndpoints.matchLabels={role: api, target: egress} and toPorts.ports=[{port: "8080", protocol: TCP}]. Do not include a DNS allow. The dns-blocked client's request to the egress server by IP must return 200, and the request by Service name must time out.

The server egress Pod is the control for port 8080. If the direct IP path is also blocked just because DNS is missing, you must also check the selector or the direction.

Restore only DNS on a different comparison client

In /root/cca-policy/dns-open.yaml, write and apply the dns-open CNP in cca-blue. It selects client-mode=open. The first egress entry is the API target and TCP 8080 from the previous step as they are. The toEndpoints.matchLabels of the second entry are k8s:io.kubernetes.pod.namespace=kube-system and k8s:k8s-app=kube-dns, and toPorts.ports has two entries, UDP and TCP, of the string 53. Both the IP and the name access of dns-open must return 200. Leave the restriction of dns-blocked as it is.

Instead of opening all external connections, allow only the name resolution endpoint and port. Check the selector of the client used for the restoration.

When you add an allow, a door opens to the other team

Preserve the initial blue-union CNP. In /root/cca-policy/union.yaml, write and apply the red-union NetworkPolicy in cca-blue. Select target=union and set policyTypes=[Ingress]. In the single from of the single ingress, put namespaceSelector.matchLabels.team=cca-red and podSelector.matchLabels.role=client together. The blue client and the red client must get 200, and the stranger must time out.

The two conditions are bundled in the same peer, but the allows of the existing CNP and the new NP are combined as separate paths.

Make an explicit deny take precedence over allow policies

Preserve the initial blue-deny CNP and red-deny NetworkPolicy. In /root/cca-policy/deny.yaml, write and apply the deny-red CNP in cca-blue. Select target=deny, and in the single fromEndpoints of the single ingressDeny, put k8s:io.kubernetes.pod.namespace=cca-red and k8s:role=client together in matchLabels. The blue client must get 200, and the stranger and the red client must time out.

If you block the red team by deleting the allow policies, the comparison of explicit deny precedence disappears. Keep the two initial allows.

An incident report from the current policies and the flows of the same requests

Save the result of the tool's report command to /root/cca-policy/report.json. The three requests the red client sent to baseline, union, and deny must be 200, 200, and timeout respectively. From the output, compare the current Pod UID/IP/identity, the policy UID/spec hash, the request source port, and the Hubble FORWARDED/DROPPED flows on the same node and time. If you changed a policy or a Pod, create the report again. In the final full grading, all the earlier steps must still be alive.

Do not attach just any DROPPED line. Match the TCP source port and the endpoints on both sides, and distinguish the HTTP result from the observation of packet forwarding.