Give a Machine Outside Kubernetes the Same Identity

Goal

Bring up a VM sidecar on a 'machine outside the mesh' made with a network namespace inside the same VM, put it in the mesh with a WorkloadGroup and a WorkloadEntry, and confirm that mTLS in both directions and STRICT and authorization policies apply exactly as they do to a Pod.

Why it matters

A machine that could not be moved to a container easily becomes an exception to the mesh, and exceptions lead to PERMISSIVE or wide allow rules and become holes in security. If you give that machine the same sidecar and the same identity, you can apply the same policies with no exception. The procedure is long and it is easy to get stuck on name resolution, the token, and the interception rules, so doing it once all the way through is the fastest study.

Steps

Put up the materials with kubectl apply -f /opt/fixtures/istlab/vmworkload-app.yaml and wait until the Pods are ready. This VM has a network namespace legacy (192.168.77.2), and an app runs inside it on 8080. Write three lines into /root/istlab-vm/01-outside.txt — the status code of http://192.168.77.2:8080/ from the client from_client=, the status code of the same request from the probe outside the mesh from_outside=, and in_mesh=, which is yes if legacy appears in istioctl proxy-status and no otherwise.
Write four resources into /root/istlab-vm/vmns.yaml and apply them — the namespace vmns (with the injection label), the service account legacy-sa, the WorkloadGroup legacy (metadata label app: legacy, the template's service account legacy-sa, port http: 8080), and the Service legacy that exposes 8080 with selector app: legacy (port name http).
Save kubectl -n vmns get workloadgroup legacy -o yaml into /root/istlab-vm/wg.yaml, and make the files for the VM with istioctl x workload entry configure -f /root/istlab-vm/wg.yaml -o /root/istlab-vm/files --clusterID Kubernetes --ingressIP <istiod 서비스의 ClusterIP> --internalIP 192.168.77.2 (the placeholder is the istiod Service's ClusterIP). cluster.env, istio-token, mesh.yaml, root-cert.pem, and hosts must be created in /root/istlab-vm/files.
Put the files from step 3 in the places the official docs give (root-cert.pem to /etc/certs/root-cert.pem, istio-token to /var/run/secrets/tokens/istio-token, cluster.env to /var/lib/istio/envoy/cluster.env, mesh.yaml to /etc/istio/config/mesh, and the lines of hosts appended to /etc/netns/legacy/hosts for the namespace), change the owner to istio-proxy, and then bring up /usr/local/bin/istio-start.sh inside the namespace (working directory /, POD_NAME=legacy-vm). legacy-vm.vmns must appear in istioctl proxy-status, and write two lines into /root/istlab-vm/04-iptables.txt: netns_backend= (the side that holds the ISTIO rules inside the namespace: legacy or nft) and host_istio_rules= (the number of ISTIO rules in the host's nat table — both nft and legacy).
Write the WorkloadEntry legacy-vm (namespace vmns, address 192.168.77.2, label app: legacy, service account legacy-sa) into /root/istlab-vm/workloadentry.yaml and apply it. After applying, http://legacy.vmns:8080/ from the client must return the body legacy-vm and 200.
Call http://web.shop/ from inside the namespace with an x-request-id attached, and save, exactly as it is, the one access log line of the web sidecar that received that request in /root/istlab-vm/06-from-vm.log.
Write two resources into /root/istlab-vm/secure.yaml and apply them — the PeerAuthentication strict (STRICT) of vmns and the AuthorizationPolicy legacy-only-client (selector app: legacy, ALLOW, principal cluster.local/ns/shop/sa/client). After applying, the client must get 200, the intruder in other must get 403, and if the probe outside the mesh calls 192.168.77.2:8080 in plain text, the connection must be dropped.
Write five lines into /root/istlab-vm/08-report.md — vm_identity= (the SPIFFE URI of the certificate the VM sidecar received), netns_iptables_backend= and host_istio_rules= (step 4), plain_from_outside= (allowed if the probe's plain request goes through now, rejected if it is dropped), and intruder= (the intruder's status code now) — and write what you learned below that in at least four lines.

Notes

This VM takes 3–5 minutes to prepare. The network namespace legacy (192.168.77.2), the app inside it (8080), and the sidecar package for VMs (istio-sidecar.deb 1.31.0) are already prepared by the recipe.
You type commands inside the namespace with ip netns exec legacy <명령> (the placeholder is the command). The /etc/hosts and /etc/resolv.conf inside it are the files in /etc/netns/legacy/.
The sidecar log is /var/log/istio/istio.log and the start log is /var/log/istio/start.log. When you bring it up again, first shut down the pilot-agent inside the namespace (find it with ip netns pids legacy).
In this VM, the istio-token expires if the session goes past an hour. Finish through step 4 within an hour.
Common mistake — overwriting /etc/netns/legacy/hosts wholesale and erasing the host name line. istio-start.sh's sudo stalls for tens of seconds waiting for name resolution.

How does the machine outside the mesh look right now

Put up the materials with kubectl apply -f /opt/fixtures/istlab/vmworkload-app.yaml and wait until the Pods are ready. This VM has a network namespace legacy (192.168.77.2), and an app runs inside it on 8080. Write three lines into /root/istlab-vm/01-outside.txt — the status code of http://192.168.77.2:8080/ from the client from_client=, the status code of the same request from the probe outside the mesh from_outside=, and in_mesh=, which is yes if legacy appears in istioctl proxy-status and no otherwise.

A network namespace is a network stack the kernel gives separately — the addresses, routing table, and iptables are all separate, so even inside the same VM it acts like 'another machine'. It is connected to the host by a veth pair (192.168.77.1 and 192.168.77.2), and traffic going from a Pod to that address is routed by the host. When you look inside, use ip netns exec legacy <명령> (the placeholder is the command). Right now this machine is an address the mesh does not know, so the client's sidecar just lets it flow out.

The template for the VM — WorkloadGroup and a Service

Write four resources into /root/istlab-vm/vmns.yaml and apply them — the namespace vmns (with the injection label), the service account legacy-sa, the WorkloadGroup legacy (metadata label app: legacy, the template's service account legacy-sa, port http: 8080), and the Service legacy that exposes 8080 with selector app: legacy (port name http).

A WorkloadGroup is a Pod template for VMs — like labels, a service account, and ports, it says 'machines like this open this port with this identity'. A Kubernetes Service's selector picks not only Pods but also Istio's WorkloadEntries, so you can put a VM behind an existing service name. There is not yet a single VM (WorkloadEntry), so the service's endpoints are empty.

Make the files to hand to the VM

Save kubectl -n vmns get workloadgroup legacy -o yaml into /root/istlab-vm/wg.yaml, and make the files for the VM with istioctl x workload entry configure -f /root/istlab-vm/wg.yaml -o /root/istlab-vm/files --clusterID Kubernetes --ingressIP <istiod 서비스의 ClusterIP> --internalIP 192.168.77.2 (the placeholder is the istiod Service's ClusterIP). cluster.env, istio-token, mesh.yaml, root-cert.pem, and hosts must be created in /root/istlab-vm/files.

The VM's sidecar needs to know five things — who it is (the namespace, service account, and labels in cluster.env), the token to present when it first receives a certificate (istio-token, a service account token), the mesh configuration (mesh.yaml), the root to trust (root-cert.pem), and the address to find istiod (hosts). For an installation spanning several networks you would give the address of the east-west gateway, but in this VM the network namespace reaches istiod's ClusterIP directly through the host's kube-proxy rules. You get the istiod address with kubectl -n istio-system get svc istiod -o jsonpath='{.spec.clusterIP}'.

Bring up the sidecar on the machine outside the mesh

Put the files from step 3 in the places the official docs give (root-cert.pem to /etc/certs/root-cert.pem, istio-token to /var/run/secrets/tokens/istio-token, cluster.env to /var/lib/istio/envoy/cluster.env, mesh.yaml to /etc/istio/config/mesh, and the lines of hosts appended to /etc/netns/legacy/hosts for the namespace), change the owner to istio-proxy, and then bring up /usr/local/bin/istio-start.sh inside the namespace (working directory /, POD_NAME=legacy-vm). legacy-vm.vmns must appear in istioctl proxy-status, and write two lines into /root/istlab-vm/04-iptables.txt: netns_backend= (the side that holds the ISTIO rules inside the namespace: legacy or nft) and host_istio_rules= (the number of ISTIO rules in the host's nat table — both nft and legacy).

istio-sidecar.deb (the recipe already installed it) puts in pilot-agent, envoy, and istio-start.sh. istio-start.sh plants interception rules with iptables and starts pilot-agent as the istio-proxy user. If you run this inside the namespace with ip netns exec legacy setsid --fork nohup env POD_NAME=legacy-vm /usr/local/bin/istio-start.sh > 로그 2>&1 </dev/null (the word after > is the log file), the rules go only into that namespace's iptables. The script uses relative paths such as ./var/lib/istio, so bring it up after cd /. You look at the rules with ip netns exec legacy iptables-legacy -t nat -S and iptables-nft, and on the host side you look without a namespace. The log is /var/log/istio/istio.log.

Put one VM behind a service — WorkloadEntry

Write the WorkloadEntry legacy-vm (namespace vmns, address 192.168.77.2, label app: legacy, service account legacy-sa) into /root/istlab-vm/workloadentry.yaml and apply it. After applying, http://legacy.vmns:8080/ from the client must return the body legacy-vm and 200.

A WorkloadEntry is a resource that represents one VM like a Pod. The labels match the selector of the service legacy, so istiod pushes this address down to the sidecars as that service's endpoint. The client's sidecar communicates with the VM's sidecar over mTLS, and on the incoming side, the iptables inside the namespace redirects 8080 to the VM sidecar. 192.168.77.2 must appear in istioctl proxy-config endpoints client.shop --cluster 'outbound|8080||legacy.vmns.svc.cluster.local'. Auto-registration is turned off in this installation, so you make it by hand.

Call an in-mesh service by name from the VM

Call http://web.shop/ from inside the namespace with an x-request-id attached, and save, exactly as it is, the one access log line of the web sidecar that received that request in /root/istlab-vm/06-from-vm.log.

The VM's sidecar also has the DNS proxy turned on (ISTIO_META_DNS_CAPTURE in cluster.env), so it resolves cluster service names, and the sidecar receives the outgoing request and sends it over mTLS. In the web-side log, see that the source of that request is 192.168.77.2 and the SNI is outbound_.80_._.web.shop.svc.cluster.local — it is evidence that it was a connection inside the mesh and not plain text. To call, it is ip netns exec legacy curl ….

Put mTLS and authorization in front of the VM too

Write two resources into /root/istlab-vm/secure.yaml and apply them — the PeerAuthentication strict (STRICT) of vmns and the AuthorizationPolicy legacy-only-client (selector app: legacy, ALLOW, principal cluster.local/ns/shop/sa/client). After applying, the client must get 200, the intruder in other must get 403, and if the probe outside the mesh calls 192.168.77.2:8080 in plain text, the connection must be dropped.

PeerAuthentication and AuthorizationPolicy pick workloads by label, whether it is a Pod or a VM. The VM's sidecar attached to istiod carrying the label in cluster.env (app: legacy), so the same policy goes down to the VM's sidecar. A request coming in as plain text from outside the mesh is redirected to the sidecar by the iptables inside the namespace, and the STRICT sidecar cuts that connection. It takes a few seconds for the policy to spread.

Write down the route the machine outside the mesh took into the mesh

Write five lines into /root/istlab-vm/08-report.md — vm_identity= (the SPIFFE URI of the certificate the VM sidecar received), netns_iptables_backend= and host_istio_rules= (step 4), plain_from_outside= (allowed if the probe's plain request goes through now, rejected if it is dropped), and intruder= (the intruder's status code now) — and write what you learned below that in at least four lines.

You can see the VM sidecar's certificate inside the namespace through Envoy's admin port — decode the default certificate of ip netns exec legacy curl -s 'localhost:15000/config_dump?resource=dynamic_active_secrets' from base64 and read it with openssl x509 -noout -ext subjectAltName (since it is not a Pod, istioctl proxy-config cannot attach to this proxy).