Building Up a Zero-Trust Policy

Goal

While building up mTLS enforcement and authorization policies in order, you internalize through manifests the evaluation order (CUSTOM → DENY → ALLOW) and the property that "if even one ALLOW exists, it flips to default deny."

Why it matters

Mesh security adoption fails almost always because steps are skipped. If you turn on STRICT without cleaning up plaintext sources, clients without sidecars are cut off immediately, and if you lay down default deny without knowing the call graph, an outage occurs without knowing what to open.

There is one fact that is especially easy to forget. principals come from the mTLS certificate, so a plaintext request in PERMISSIVE has an empty principal, and then it matches no principals condition at all. This is the most common cause of "the policy is clearly right, yet 403." That is why this lab follows the order mTLS → default deny → least privilege → safety net → end-user authentication as is.

This lab is a kwok-based lab on designing API objects and manifests. Kubernetes objects are stored in the API, but no real payments container, Envoy, or JWT verification server runs. You write the Istio policies as files under /root/ica-security/, and HTTP responses and encryption behavior are not graded. You do not connect to the example IdP address.

Step 8 is a design that keeps the existing ALLOW and applies a DENY to requests without a JWT on port 8080. A separate JWT ALLOW creates not an AND of the two conditions but an OR of allowed paths, so it is not used. It is not a policy that guarantees mandatory JWT on other ports or allowing metrics access.

Steps

Create the namespace ica-security and attach the label istio-injection=enabled.
In the namespace ica-security, create the service accounts payments-api and orders-api.
In the same namespace, deploy a Deployment payments-api. The Pod label is app=payments-api, serviceAccountName is payments-api, and the image is nginx:1.27-alpine. Then create a Service payments-api with two ports, 8080 (name http) and 9090 (name http-metrics).
In /root/ica-security/pa-mesh.yaml, write a PeerAuthentication. The name is default, the namespace is istio-system, mtls.mode: STRICT, and no selector. Next, in /root/ica-security/pa-workload.yaml, write a policy with namespace ica-security, selector app: payments-api, mtls.mode: STRICT, and with portLevelMtls making only port 9090 PERMISSIVE.
In /root/ica-security/ap-deny-all.yaml, write an AuthorizationPolicy. The name is default-deny, the namespace is ica-security, and spec is an empty object.
In /root/ica-security/ap-allow-orders.yaml, write an AuthorizationPolicy. Selector app: payments-api, action: ALLOW, and in one rule, from.source.principals is cluster.local/ns/ica-security/sa/orders-api, to.operation.methods is POST, and to.operation.paths is /v1/charges.
In /root/ica-security/ap-deny-admin.yaml, write an AuthorizationPolicy. Selector app: payments-api, action: DENY, and the rule is only /admin/* in to.operation.paths, with no from. Attach istio.io/dry-run: "true" to metadata.annotations.
In /root/ica-security/ra-jwt.yaml, write a RequestAuthentication. The name is payments-api-jwt, the namespace is ica-security, the selector is app: payments-api, there is one jwtRule, the issuer is https://idp.example.com/, the jwksUri is https://idp.example.com/.well-known/jwks.json, and the audiences are [payments-api]. Next, in /root/ica-security/ap-require-jwt.yaml, write an AuthorizationPolicy named require-jwt, with the same namespace and selector, and action: DENY. In one rule, from.source.notRequestPrincipals is ["*"], and to.operation.ports of the same rule is ["8080"]. Do not put additional from, to, or rules conditions, or a dry-run. Keep the least-privilege ALLOW from step 6 as it is.

Notes

When you write a SPIFFE ID in principals, omit the spiffe:// prefix and write it in the form cluster.local/ns/.../sa/....
notRequestPrincipals: ["*"] selects requests that have no verified JWT principal. DENY is evaluated before ALLOW.
When you use HTTP attributes in a DENY, missing attributes of TCP requests can also match. This task limits the target port to 8080.
Putting principals and requestPrincipals together inside the same source is also a valid AND design. Distinguish it from splitting them across separate ALLOW policies.
Common mistake 1: leaving out the spec key itself when creating default deny. Then the policy is not interpreted as intended.
Common mistake 2: attaching a selector to the mesh-wide PeerAuthentication. At that moment it becomes a workload policy, not a mesh-wide one.
The dry-run annotation value is the string "true". If you write it without quotes it becomes a boolean, which violates the annotation rules.

Creating the security lab namespace

Create the namespace ica-security and attach the label istio-injection=enabled.

Without a sidecar, you can apply neither mTLS nor authorization. Attach the automatic injection label.

Creating a dedicated service account per workload

In the namespace ica-security, create the service accounts payments-api and orders-api.

The last field of a SPIFFE ID is the service account. If several workloads share default, you cannot tell them apart in policy.

Deploying a workload with its service account attached

In the same namespace, deploy a Deployment payments-api. The Pod label is app=payments-api, serviceAccountName is payments-api, and the image is nginx:1.27-alpine. Then create a Service payments-api with two ports, 8080 (name http) and 9090 (name http-metrics).

You must state serviceAccountName explicitly in the Pod spec for a certificate to be issued for that identity. Open both the application port and the metrics port on the Service.

Mesh-wide STRICT and a port exception

In /root/ica-security/pa-mesh.yaml, write a PeerAuthentication. The name is default, the namespace is istio-system, mtls.mode: STRICT, and no selector. Next, in /root/ica-security/pa-workload.yaml, write a policy with namespace ica-security, selector app: payments-api, mtls.mode: STRICT, and with portLevelMtls making only port 9090 PERMISSIVE.

A mesh-wide policy is placed in the root namespace with the fixed name and has no selector. The moment a selector is attached, it becomes a workload policy. Specify the exception port narrowly with portLevelMtls.

Creating a namespace default deny

In /root/ica-security/ap-deny-all.yaml, write an AuthorizationPolicy. The name is default-deny, the namespace is ica-security, and spec is an empty object.

An ALLOW policy with no rules at all is a total deny. You must not leave out the spec field itself; leave it in an empty state.

Opening only along the call graph

In /root/ica-security/ap-allow-orders.yaml, write an AuthorizationPolicy. Selector app: payments-api, action: ALLOW, and in one rule, from.source.principals is cluster.local/ns/ica-security/sa/orders-api, to.operation.methods is POST, and to.operation.paths is /v1/charges.

principals is a path format that starts from the trust domain. Methods and paths go under to.operation. In this step, use only the service account condition.

Blocking the admin path with one more layer, a DENY

In /root/ica-security/ap-deny-admin.yaml, write an AuthorizationPolicy. Selector app: payments-api, action: DENY, and the rule is only /admin/* in to.operation.paths, with no from. Attach istio.io/dry-run: "true" to metadata.annotations.

DENY is evaluated before ALLOW, so it serves as a safety net that defends against ALLOW mistakes. It must block regardless of the source, so do not put a from. Since this is a stage before putting it into production, also attach the shadow-evaluation annotation.

Making the end-user token mandatory

In /root/ica-security/ra-jwt.yaml, write a RequestAuthentication. The name is payments-api-jwt, the namespace is ica-security, the selector is app: payments-api, there is one jwtRule, the issuer is https://idp.example.com/, the jwksUri is https://idp.example.com/.well-known/jwks.json, and the audiences are [payments-api]. Next, in /root/ica-security/ap-require-jwt.yaml, write an AuthorizationPolicy named require-jwt, with the same namespace and selector, and action: DENY. In one rule, from.source.notRequestPrincipals is ["*"], and to.operation.ports of the same rule is ["8080"]. Do not put additional from, to, or rules conditions, or a dry-run. Keep the least-privilege ALLOW from step 6 as it is.

RequestAuthentication is token verification. A separate ALLOW merges with the existing allowance, so it is not making it mandatory. Reject requests without a JWT principal first with a DENY, and limit the target port to the string 8080.