You Cannot Trust a Pod IP — the Problem a Service Solves

One-line summary

A Pod can die at any time and is reborn with a new IP. A Service puts a name and an address that do not die in front, and automatically refreshes the list of Pods behind it with a label selector. Storage follows the same way of thinking: you declare not "which disk" but "what kind of storage, and how much, is needed."

Why this was needed

As we saw in the previous module, if you delete a Pod, a Pod with a different name and a different IP comes. Then what should the side that was connecting to that Pod do? If it had to look up the list again and re-establish connections every time, every application would have to become a Kubernetes API client.

A Service solves this problem with a layer of indirection. It sets up a stable virtual IP (ClusterIP) and a DNS name, and keeps refreshing the list of Pods behind it with the selector. The client needs to know only the name.

How it works

The three promises of the Kubernetes network model

1. Every Pod can communicate with every other Pod without NAT
2. An agent on a node can communicate with all Pods on that node
3. The IP a Pod sees as its own is the same as the IP others see for that Pod

CNI plugins differ only in how they keep these promises; the promises themselves are the same.

Services and endpoints

When you create a Service, the endpoint controller collects the IPs of Ready Pods that match the selector and writes them into an EndpointSlice (formerly Endpoints). Two things matter here.

• Only Pods that are Ready get in. This is why a Pod is taken out of traffic when its readinessProbe fails.
• If the selector is off, the endpoints become 0. No error occurs. The Service is fine and the ClusterIP is issued normally. It is just that once you connect, it goes nowhere. Checking whether the result of kubectl get endpointslice is empty is the first diagnosis for this family of outages.

For reference, since Kubernetes v1.33 the Endpoints API is deprecated, and you are advised to move to EndpointSlice. EndpointSlice can split the list into several pieces in large clusters and also supports dual-stack.

Service types

Type	Exposure scope	Where it is used
ClusterIP (default)	Inside the cluster	Communication between services
NodePort	Port 30000-32767 on every node	Development and simple external exposure
LoadBalancer	An external load balancer	Integration with a cloud/on-prem LB
ExternalName	DNS CNAME	Giving a name to a service outside the cluster

A headless Service (clusterIP: None) is the exception. It does not create a VIP and returns the Pod IPs as they are in DNS lookups. It is used when you need to address individual Pods directly, as with a StatefulSet.

DNS naming rules

<서비스>.<네임스페이스>.svc.cluster.local

Within the same namespace, <서비스> (the service name) alone is enough. This one rule makes the problem of "the address differs per environment" disappear: you just swap the namespace and the same manifest runs.

Config and secrets

ConfigMap and Secret are devices for separating the image from the environment. There are two ways to inject them: as environment variables, or mounted as a volume. When mounted as a volume, the file is updated when the value changes, but environment variables are baked in at process start and are not updated.

One fact about Secrets you must know at the KCNA level: base64 is an encoding, not encryption. With default settings, a Secret is stored in etcd in effectively plain text, and running kubectl get secret ... -o jsonpath through base64 -d once gives the original. Real encryption requires turning on an EncryptionConfiguration separately. (KCSA covers this in detail.)

Storage

• PersistentVolume (PV): The actual storage. Created by an administrator or by a provisioner.
• PersistentVolumeClaim (PVC): A user's request form. "I need 1Gi of RWO storage."
• StorageClass: A rule that looks at a request form and creates a PV automatically.

A Pod does not point at a PV directly; it points at a PVC. Thanks to this layer of indirection, the Pod manifest stays the same whether the backend is NFS or block storage.

Access modes also come up often on the exam: ReadWriteOnce (read/write from one node), ReadOnlyMany (read from many nodes), and ReadWriteMany (read/write from many nodes). Few backends support RWX, so you end up using file storage such as NFS.

What it looks like in practice

The author's homelab does not install kube-proxy at all and uses Cilium's eBPF instead for Service load balancing. And they verified that it really runs: 0 KUBE- chains in iptables on the nodes, and instead the eBPF map directly held frontend-to-backend mappings such as 10.96.0.10:53/TCP → 10.244.0.150:53/TCP and 10.96.0.1:443/TCP → 10.0.0.120:6443/TCP.

An interesting bootstrap problem comes up here. Without kube-proxy, when Cilium itself connects to the API server it cannot use the ClusterIP (10.96.0.1) of the kubernetes Service, because the one that would create that VIP is Cilium itself. So at install time you have to tell it the real address directly, with k8sServiceHost=10.0.0.120 and k8sServicePort=6443. Here the meaning of the phrase "a Service is a layer of indirection" becomes clear.

Storage in the same cluster uses csi-driver-nfs to attach a NAS and use RWX. The L4 load balancer is MetalLB and the address pool is 10.0.0.200-215. Gitea took .200, ArgoCD took .201, Harbor took .202, and Grafana took .203. It is a setup where you can see with your own eyes that a LoadBalancer-type Service really is the action of "peeling one IP off the pool and attaching it."

What you will do in the next lab

In the next lab you stand up a backend Deployment, attach a ClusterIP Service, and check that the endpoints are picked up. Then you create a Service with a deliberate selector typo to reproduce the endpoints becoming 0, and go on to check NodePort, ConfigMap/Secret injection, PVC requests, and DNS naming rules by hand.