Scheduler, kube-proxy, Container Runtime — Credentials of the Quiet Components

In one line

kube-scheduler and kube-proxy each hold a client certificate for the API server and open their own HTTPS port. The container runtime creates and deletes every container on the node through a single Unix socket. These three components sit in places that the audit log does not catch well, so you must know what they hold and where they leave doors open, and narrow those doors.

Why this was needed

KCSA's cluster component security domain does not ask only about the API server, etcd, and the kubelet. Attackers do not knock on the best-guarded door. They look for credentials and ports that administrators do not look into closely, such as the scheduler's kubeconfig, kube-proxy's metrics port, and the runtime socket. The API server bypass risks document was written from exactly this perspective — a route that can change the cluster without going through the API server passes through neither admission control nor the audit log.

How it works

kube-scheduler — a control plane process that holds binding permission

According to the kube-scheduler reference, the scheduler is a control plane process that, for each Pod in the scheduling queue, picks and ranks nodes that fit its constraints and available resources and binds it. This binding permission is the threat model. An attacker who obtains the scheduler's credentials (in kubeadm, the client certificate in /etc/kubernetes/scheduler.conf) can place a Pod on any node they want and can attach their own Pod to the same node as a sensitive workload.

The scheduler also opens its own HTTPS port. In the ports and protocols table, it is 10259. Two flags decide this port's authentication and authorization. --authentication-kubeconfig points to a kubeconfig that can create TokenReviews, and if you leave it empty, every token request is treated as anonymous. --authorization-kubeconfig points to a kubeconfig that can create SubjectAccessReviews, and if you leave it empty, every request that is not allowed to skip authorization is rejected. That is, you must leave both flags delegating to the API server so that the metrics and health endpoints do not become a door anyone can read. --leader-elect defaults to true, so only one of the replicated schedulers operates.

You must also know the routes that bypass the scheduler. The safely drain a node document says that if you specify a Pod's nodeName directly, it is tied to that node without going through the scheduler. A user with permission to create Pods can choose a node regardless of the scheduler's policies, so sensitive node isolation must be enforced not by scheduler settings but by admission (such as PodNodeSelector) and taints.

kube-proxy — a network proxy that runs on every node

The kube-proxy reference explains that kube-proxy runs on each node, reflects the API's Service definitions onto the node, and forwards TCP, UDP, and SCTP to backends. The proxy modes on Linux are iptables (the default), ipvs, and nftables. kube-proxy implements Services; it does not enforce NetworkPolicy. Policy enforcement is the job of the CNI, covered in the next reading.

There are two things to look at for security. First, credentials. The PKI certificates and requirements document says that each node has a client certificate for kube-proxy to authenticate to the API server. A process that can read this kubeconfig placed on the node can talk to the API server with kube-proxy's permissions. Second, the ports left open. The default of --healthz-bind-address is 0.0.0.0:10256, so it accepts health checks on all interfaces, but the default of --metrics-bind-address is 127.0.0.1:10249, which emits metrics only locally. If you widen this to 0.0.0.0:10249 for monitoring, anyone on the node network can read kube-proxy's internal state. You must widen it knowing why the default is local.

The container runtime — one socket is the whole node

According to the CRI document, the kubelet connects to the runtime as a gRPC client, and the endpoint is a Unix socket (/var/run/containerd/containerd.sock for containerd and /var/run/crio/crio.sock for CRI-O). The "container runtime socket" section of the API server bypass risks document states the threat — an attacker with access to this socket can start new containers or interact with running containers, and if containers on that node hold Secrets, can widen their privileges to other nodes or the control plane.

The mitigations the document presents are at the filesystem level. Restrict file access to the socket to root, isolate the kubelet from other components with kernel namespaces, forbid hostPath mounts that contain the runtime socket, whether directly or through a parent directory, keep hostPath read-only so that it cannot be used to bypass directory restrictions, and limit user access to the node itself. The "static Pod" section of the same document also ties into the runtime — a static Pod that the kubelet starts directly from the manifest directory is not managed by the API server, so an attacker who can write to that directory can put a Pod that uses hostPath on the node without going through admission. If a static Pod fails admission, the kubelet does not register it with the API server, but the Pod keeps running on the node as it is.

Runtime isolation is a tool in the opposite direction. The RuntimeClass document explains that you can choose a different runtime configuration per Pod to balance performance and security. Workloads that need a high level of information security assurance can run on a runtime that uses hardware virtualization to gain extra isolation while accepting the overhead. If you configure a handler in the node's CRI implementation and create a RuntimeClass object, you select it with runtimeClassName in the Pod spec.

What it looks like in the field

kube-proxy was opened to 0.0.0.0 for metrics collection. After widening metricsBindAddress on the grounds that Prometheus had to scrape 10249 from outside the node, any Pod on the node network could reach 10249 on the node IP without hostNetwork. If you widen it, a rule that allows only the collector with a NetworkPolicy or firewall must come with it.

A CI runner mounts the runtime socket. A Pod that put /var/run/containerd/containerd.sock in as a hostPath in order to build images is a Pod that can handle every container on the node. It is exactly why the document says to forbid this mount, and also why the baseline of the Pod Security Standards blocks hostPath.

What to look at in the next reading

In the next reading, we look at three components at the boundary — the CNI that actually enforces NetworkPolicy, storage where hostPath and CSI credentials are entangled, and the client-side risk that kubeconfig and exec plugins expose.