TT Lab
Get started
Learn Learning paths Courses

Istio Service Mesh

Bring the server you could not migrate into the mesh: WorkloadGroup and WorkloadEntry

Continue in TT Lab

Goal

You frame a group of servers outside the mesh with a WorkloadGroup, register servers one by one with WorkloadEntry, and bind them to a service name inside the mesh with a ServiceEntry. At the end, you actually create the configuration bundle that server would receive with istioctl x workload entry configure, and leave a report that counts what the selector selects.

Why it matters

Even after adopting a mesh, servers that could not be moved to Kubernetes remain. If you leave them outside, what the mesh gives is cut off there — identity, encryption, access policy, and above all a single service map. Mesh expansion is bringing that server in not as a guest but as a member. The key is identity. Inside Kubernetes a Pod gets its identity from a service account, but a VM has no such thing. So you write in the WorkloadGroup which service account to use, and put into that server in advance a token issued by the cluster and the root certificate. Only when these three match can the VM's proxy connect to the control plane and prove its identity. The relationship of the three objects is also easy to confuse, but it sorts out if you see the template for the group as WorkloadGroup, one server as WorkloadEntry, and the name to call as ServiceEntry.

Steps

  1. Create /root/ist-expansion and the namespace expansion, and create a ServiceAccount billing-sa in it. And in /root/ist-expansion/identity.txt, write on one line the identity string that a workload using this service account will receive in the mesh — the trust domain is the default cluster.local.
  2. In /root/ist-expansion/wg.yaml, write a WorkloadGroup legacy-billing and apply it to expansion — spec.metadata.labels is app: billing, spec.template is serviceAccount: billing-sa, network: onprem, and ports.http: 8080, and spec.probe is periodSeconds: 5 with httpGet looking at /healthz on port 8080.
  3. In /root/ist-expansion/wg-bad-probe.yaml, write a WorkloadGroup bad-probe but put both httpGet and tcpSocket in spec.probe. In /root/ist-expansion/wg-no-template.yaml, write no-template but leave out spec.template entirely. Do only a server-side trial application for both, and collect the rejection messages in order in /root/ist-expansion/wg-reject.txt.
  4. In /root/ist-expansion/we.yaml, write a WorkloadEntry billing-1 and apply it — metadata label app: billing, address is 10.20.30.41, network is onprem, serviceAccount is billing-sa, locality is dc1/rack2, and ports.http is 8080. Then in /root/ist-expansion/we-typo.yaml, write billing-typo but apply it with the address written with a space like 10.20.30 .41, and save the output of istioctl analyze -n expansion -o json to /root/ist-expansion/analyze-address.json. Put the label as app: billing-broken.
  5. In /root/ist-expansion/se.yaml, write a ServiceEntry billing-svc and apply it — hosts is just billing.expansion.internal, location is MESH_INTERNAL, resolution is STATIC, the port is number 8080 with the name http and protocol HTTP, and workloadSelector.labels is app: billing.
  6. Create three ServiceEntry files and check the rules with a server-side trial application — /root/ist-expansion/se-none-endpoints.yaml (resolution: NONE but with endpoints), /root/ist-expansion/se-roundrobin-two.yaml (resolution: DNS_ROUND_ROBIN with two endpoints), and /root/ist-expansion/se-roundrobin-one.yaml (resolution: DNS_ROUND_ROBIN with one endpoints). Write the results in /root/ist-expansion/resolution-rules.tsv as <파일이름>\t<accepted|rejected> (the placeholders are the file name and accepted or rejected) in the order above, and collect the rejection messages in /root/ist-expansion/resolution-reject.txt. Do not actually apply any of the three files.
  7. First create the istio-system namespace, extract only the ConfigMap istio from the istioctl manifest generate --set profile=minimal render, save it to /root/ist-expansion/istio-cm.yaml, and apply it. Then create a ConfigMap istio-ca-root-cert in expansion — the key name is root-cert.pem and the value is a self-signed root certificate made with openssl (/root/ist-expansion/root-cert.pem). Finally run istioctl x workload entry configure -f /root/ist-expansion/wg.yaml -o /root/ist-expansion/vmcfg --clusterID lab-cluster to create the configuration bundle.
  8. In /root/ist-expansion/we2.yaml, write a second server billing-2 and apply it — label app: billing, address 10.20.30.42, network: onprem, serviceAccount: billing-sa, locality: dc1/rack3, and ports.http 8080. Then create /root/ist-expansion/se-match.sh so that for each ServiceEntry in expansion that has a workloadSelector, it prints <ServiceEntry 이름>\t<잡힌 WorkloadEntry 수>\t<이름들 쉼표> (the placeholders are the ServiceEntry name, the number of WorkloadEntries selected, and the names joined by commas), in name order, only to standard output, and save that output to /root/ist-expansion/match-result.txt.

Notes

First decide what the mesh will call that server

Create /root/ist-expansion and the namespace expansion, and create a ServiceAccount billing-sa in it. And in /root/ist-expansion/identity.txt, write on one line the identity string that a workload using this service account will receive in the mesh — the trust domain is the default cluster.local.

A mesh identity comes not from an IP or host name but from a service account. The format is spiffe://<신뢰도메인>/ns/<네임스페이스>/sa/<서비스어카운트> (the placeholders are the trust domain, the namespace, and the service account). A server outside the mesh too must receive this label to come in.

Make a template for a group of VMs, not one VM

In /root/ist-expansion/wg.yaml, write a WorkloadGroup legacy-billing and apply it to expansion — spec.metadata.labels is app: billing, spec.template is serviceAccount: billing-sa, network: onprem, and ports.http: 8080, and spec.probe is periodSeconds: 5 with httpGet looking at /healthz on port 8080.

WorkloadGroup is the place that corresponds to a Pod's Deployment — you write not an individual server but the labels, identity, network, ports, and health check method that the group has in common. The network value points to which network that server is in, and later the proxy picks paths with that value.

Take directly the two things the schema rejects

In /root/ist-expansion/wg-bad-probe.yaml, write a WorkloadGroup bad-probe but put both httpGet and tcpSocket in spec.probe. In /root/ist-expansion/wg-no-template.yaml, write no-template but leave out spec.template entirely. Do only a server-side trial application for both, and collect the rejection messages in order in /root/ist-expansion/wg-reject.txt.

A probe is set up so that you choose only one of the check methods — it is a oneOf constraint in the schema. template is required, because without it the group's defaults cannot be known. Read which field the rejection message points to.

Actually register one server, and see where a typo gets caught

In /root/ist-expansion/we.yaml, write a WorkloadEntry billing-1 and apply it — metadata label app: billing, address is 10.20.30.41, network is onprem, serviceAccount is billing-sa, locality is dc1/rack2, and ports.http is 8080. Then in /root/ist-expansion/we-typo.yaml, write billing-typo but apply it with the address written with a space like 10.20.30 .41, and save the output of istioctl analyze -n expansion -o json to /root/ist-expansion/analyze-address.json. Put the label as app: billing-broken.

A WorkloadEntry is one server in the group. The CRD schema does not check the address format, so strange values are created too — what catches them is the analyzer. Check which code it comes out under.

Bind the registered server to a service name in the mesh

In /root/ist-expansion/se.yaml, write a ServiceEntry billing-svc and apply it — hosts is just billing.expansion.internal, location is MESH_INTERNAL, resolution is STATIC, the port is number 8080 with the name http and protocol HTTP, and workloadSelector.labels is app: billing.

If you create only a WorkloadEntry, there is no name to call it by. A ServiceEntry creates the service name inside the mesh and selects with workloadSelector the workloads that will stand behind that name. If location is MESH_INTERNAL, it is treated as a mesh member and becomes a target to which mTLS and policies apply.

Each resolution has different endpoint rules

Create three ServiceEntry files and check the rules with a server-side trial application — /root/ist-expansion/se-none-endpoints.yaml (resolution: NONE but with endpoints), /root/ist-expansion/se-roundrobin-two.yaml (resolution: DNS_ROUND_ROBIN with two endpoints), and /root/ist-expansion/se-roundrobin-one.yaml (resolution: DNS_ROUND_ROBIN with one endpoints). Write the results in /root/ist-expansion/resolution-rules.tsv as <파일이름>\t<accepted|rejected> (the placeholders are the file name and accepted or rejected) in the order above, and collect the rejection messages in /root/ist-expansion/resolution-reject.txt. Do not actually apply any of the three files.

resolution decides how to find the destination — NONE uses the original destination IP as it is, STATIC uses the endpoints you wrote, and DNS resolves the name and uses it. So how many endpoints you can write differs for each way. This rule is enforced not by istioctl but by the CRD's validation rules.

Actually create the configuration bundle that server would receive

First create the istio-system namespace, extract only the ConfigMap istio from the istioctl manifest generate --set profile=minimal render, save it to /root/ist-expansion/istio-cm.yaml, and apply it. Then create a ConfigMap istio-ca-root-cert in expansion — the key name is root-cert.pem and the value is a self-signed root certificate made with openssl (/root/ist-expansion/root-cert.pem). Finally run istioctl x workload entry configure -f /root/ist-expansion/wg.yaml -o /root/ist-expansion/vmcfg --clusterID lab-cluster to create the configuration bundle.

The VM-side proxy needs three things — the mesh configuration (where to connect), the root certificate (whom to trust), and a token (who it is). The first two come from the cluster's ConfigMaps, and the token is issued directly by this command for the service account. The issued token has a short lifetime, so do not print it on the screen.

Count what the selector actually selects and leave it as a report

In /root/ist-expansion/we2.yaml, write a second server billing-2 and apply it — label app: billing, address 10.20.30.42, network: onprem, serviceAccount: billing-sa, locality: dc1/rack3, and ports.http 8080. Then create /root/ist-expansion/se-match.sh so that for each ServiceEntry in expansion that has a workloadSelector, it prints <ServiceEntry 이름>\t<잡힌 WorkloadEntry 수>\t<이름들 쉼표> (the placeholders are the ServiceEntry name, the number of WorkloadEntries selected, and the names joined by commas), in name order, only to standard output, and save that output to /root/ist-expansion/match-result.txt.

You can ask about a label selector as it is, like kubectl get workloadentry -l app=billing. If the selector has several labels, join them with commas. If you make it print 0 and a hyphen when nothing is selected, you will see it right away on the day the labels go out of step.