Reading Cilium: An Architecture Where the Agent Populates BPF Maps and Kernel Hooks Rely Solely on Those Maps

Status: Draft (2026-10-17), partially measured. Baseline version: cilium/cilium v1.20.1 (commit 7d68cfb394f2). Since a small cluster is running this exact version (agents, operator-generic, and hubble-relay v1.20.1, alongside the cilium-envoy 1.37 family), measurements were performed strictly as read-only on that cluster on 2026-10-17 (running cilium-dbg·bpftool·hubble observe read commands, reading ConfigMaps and policy counts, and reading relay logs on a single agent pod). Connection results for DNS patterns cite records (PR draft) measured between 2026-10-16 and 17 while preparing contributions. Writing convention: measured data is the baseline; unmeasured items are marked as "Unmeasured", and unverified items are marked as "Unverified".

Table of Contents

• 1. One-Sentence Summary
• 2. Architecture Diagram
• 3. Code Map
• 4. Tracing Core Paths
  • 4.1 How the Agent Compiles and Loads BPF
  • 4.2 Replacing kube-proxy: From Service connect() to Backend Resolution
  • 4.3 How toFQDNs Policies Pass Through the DNS Proxy to "Allow This IP"
  • 4.4 Hubble: From the Perf Ring Buffer to Relay
• 5. Connection to User Contributions
• 6. Measurements
• 7. Where to Start Reading
• 8. Good Places for First Contributions
• 9. Limitations
• References

1. One-Sentence Summary

To summarize Cilium in one sentence: "An office (agent) on each node updates rulebooks (BPF maps), and checkpoints (BPF programs) across the kernel decide on passing, dropping, or address translation by looking solely at the rulebooks." Checkpoints do not call the office. Because going up to user space for every single packet is slow, everything needed for decisions is written into maps in advance. Therefore, the first question to ask when reading Cilium code is always: "Who writes this information to which map, and which hook reads it?"

2. Architecture Diagram

• The two boxes above are singular per cluster. The agent watches the API server directly . Since it does not go through the operator, the dataplane of already running nodes remains unaffected even if the operator is temporarily stopped.
• The large box in the middle represents a single node. The six chips inside the agent box are the components we will follow in this post, and the cylinders at the bottom are the BPF maps used by those components.
• The table at the very bottom shows "Hook Location → Attached Program". The filename represents the attachment location. bpf_lxc.c is on the pod side, bpf_host.c is on the host side, bpf_overlay.c is the tunnel device, and bpf_sock.c represents socket system calls. The dotted boxes (XDP) are disabled in the small cluster.
• The dashed arrow on the left illustrates the path where events emitted by BPF to cilium_events perf maps ascend to the Hubble observer inside the agent (4.4).

3. Code Map

The repository is large, but the starting places are clearly defined.

Both the agent and the operator are assembled using a dependency injection framework called hive . Each feature is a "cell", and cells declare only what they provide and what they require. Agent main is daemon/main.go:11 in hive.New(cmd.Agent) is a single line, and that Agent is divided into daemon/cmd/cells.go:103 three bundles in Infrastructure·ControlPlane·datapath.Cell . ControlPlane(cells.go:194) The comment explicitly states the reason for this division. The control plane is "pure business logic," so the goal is to allow integration tests to run without kernel privileges. The actual work of handling BPF was pushed into the pkg/datapath/cells.go:52 module of datapath . The measured module tree of cilium-dbg status --verbose was also split into controlplane / datapath / infra three parts (Section 6).

4. Following the Core Path

4.1 How the Agent Compiles and Loads BPF

Cilium does not pre-bake BPF objects into the image; it compiles them with clang on the node(the agent image includes clang 19.1.7, as verified). Since kernel features and configurations (such as tunneling, masquerading, and IPv6) differ per node, unused features must be #ifdef completely removed via preprocessor definitions so that the program passes the verifier and does not hit the instruction limit.

1. -based datapath. pkg/datapath/orchestrator/orchestrator.go:167 reconciler calls Reinitialize whenever node settings change, and pkg/datapath/loader/base.go:298 Reinitialize compiles bpf_overlay.cfor tunnel devices, XDP, and—if socket LB is enabled— bpf_sock.c , attaching them to the (base.go:406) cgroup (pkg/socketlb/socketlb.go:83 Enable).
2. per endpoint (pod). pkg/datapath/loader/endpoint.go:83 ReloadDatapath first calls cache.go:176 fetchOrCompile . This is the core of the design. It creates a hash using the endpoint configuration minus the values that differ per pod (such as endpoint ID, IP, and ifindex),and if an object with the same hash already exists, it reuses it via copy instead of recompiling. The actual compilation is performed by compile.go:160 compile , which runs clang.
3. Injecting Values and Attaching. endpoint.go:195 reloadEndpoint loads the template while overwriting pod-specific values as constants(Constants: endpointConfiguration(...), inserts the policy program into the tail call map first, and then attaches cil_from_container to the ingress of the veth. The sequence is explained in the comments because opening the ingress first allows packets to enter a half-prepared program. The pod-side egress program (cil_to_container) is attached only when RequireEgressProg() is true.

Measurements show why this matters. Thirteen endpoints on a single node shared a single template hash, and there were only two templates: bpf_lxc.o(approx. 1.0MB) and bpf_host.o(approx. 1.4MB). Compilation happens once, and subsequent loads only swap constants, removing clang from pod startup latency.

4.2 Replacing kube-proxy: From Service to connect() Backends

kube-proxy translates Services into iptables (or IPVS) rules. Cilium places the same information into BPF hash maps and, whenever possible, changes the socket destination address before packets are even created . In v1.18, this control plane was rewritten to be based on StateDB, and its architecture matches the diagram in pkg/loadbalancer/README.md .

1. Reflection. pkg/loadbalancer/reflectors/k8s.go:208 runServiceEndpointsReflector aggregates Service and EndpointSlice changes briefly and writes them in a single transaction (to avoid modifying maps thousands of times during massive rollouts).
2. Writing to Tables. pkg/loadbalancer/writer/writer.go:326 UpsertServiceAndFrontends and others reconcile references across the three tables: service, frontend, and backend. Even with multiple sources, updates flow exclusively through this Writer.
3. Matching via Maps. pkg/loadbalancer/reconciler/bpf_reconciler.go:725 Update watches the frontend table and writes to the map at updateFrontend(line 842) . In map cilium_lb4_services_v2(bpf/lib/lb.h:284, whose key is (주소, 포트, 프로토콜, 슬롯) , slot 0 acts as the "head" (backend count and flags), and slots 1..n contain backend IDs. Retries are handled by the reconciler so that data sources do not need to worry about map errors.
4. Resolving at the Socket. bpf/bpf_sock.c:467 cil_sock4_connect calls connect4 in the cgroup __sock4_xlate_fwdhook (line 298) , which finds the head using lb4_lookup_service and selects a slot via sock_select_slot() % count + 1 to rewrite the socket address to the backend. The application believes it connected to the ClusterIP, but the kernel socket is bound to the backend from the start, eliminating the need for per-packet NAT.

However, there is a catch in the fourth step.

	if (is_defined(ENABLE_SOCKET_LB_HOST_ONLY) && !in_hostns)
		return -ENXIO;

	if (!udp_only && !sock_proto_enabled(protocol))
		return -ENOTSUP;

	/* In case a direct match fails, we try to look-up surrogate
	 * service entries via wildcarded lookup for NodePort and
	 * HostPort services.
	 */
	svc = lb4_lookup_service(&key, true);
	if (!svc)
		svc = sock4_wildcard_lookup_full(&key, in_hostns);
	if (!svc)
		return -ENXIO;

Source: cilium v1.20.1 bpf/bpf_sock.c lines 322–335, GPL-2.0-only OR BSD-2-Clause

socketLB.hostNamespaceOnly When enabled, sockets in the pod namespace simply return here. Who handles service address translation then? bpf_lxc.c decides this at compile time.

#if !defined(ENABLE_SOCKET_LB_FULL) || \
    defined(ENABLE_SOCKET_LB_HOST_ONLY) || \
    defined(ENABLE_L7_LB)               || \
    defined(ENABLE_SCTP)                || \
    defined(ENABLE_CLUSTER_AWARE_ADDRESSING)
# define ENABLE_PER_PACKET_LB 1
#endif

Source: cilium v1.20.1 bpf/bpf_lxc.c lines 90–96, GPL-2.0-only OR BSD-2-Clause

If there is even a single case where resolution cannot happen entirely at the socket, per-packet LB (__per_packet_lb_svc_xlate_4, line 152)is inserted into the pod veth's tc program. Official documentation explains that this bypass is intended for cases where the original ClusterIP is required (such as Istio sidecars) or for pods where socket LB is ineffective, such as KubeVirt, Kata, or gVisor (kubeproxy-free documentation "Socket LoadBalancer Bypass in Pod Namespace"). The measured small cluster was precisely in this mode (Socket LB Coverage: Hostns-only). Since the TT Lab cluster runs KubeVirt VMs, this choice is justified (KubeVirt Practice VM Series). Consequently, connections from pods to services are resolved within cil_from_container , and the socket reverse-translation map (cilium_lb4_reverse_sk) contained only 13 host namespace sockets. Assuming "kube-proxy replacement = socket LB" across the board is a common misconception.

4.3 How toFQDNs Policies Route Through the DNS Proxy to Allow Specific IPs

BPF only sees IPs and ports. However, users want to write policies using names like *.docker.com . Cilium's answer is It intercepts the DNS response, learns the name-to-IP mapping, and attaches the identity derived from the name to the learned IP.The name does not go down to BPF; BPF simply asks, as it always does, "Is this identity allowed?"

1. Selector registration. When the policy has toFQDNs , pkg/fqdn/namemanager/manager.go:140 RegisterFQDNSelector receives the selector, and if there is already a matching name in the cache, it immediately exposes the IP. The regular expression for the selector is pkg/policy/api/fqdn.go:115 ToRegex created via matchpattern by
2. Query interception. Because of the rules.dnsin the same policy (port 53), bpf_lxc redirects that pod's DNS packets to the node's DNS proxy. pkg/fqdn/dnsproxy/proxy.go:895 ServeDNS uses CheckAllowed(line 763) to inspect the query name. That regex is GeneratePattern(line 1439) built by combining the patterns of rules.dns with | . If it does not match, a refusal response is returned (small cluster settings are refused).
3. Response → Name cache. If allowed, it is sent upward, and before returning the response to the client upon receiving it is called NotifyOnDNSMsg . This goes through pkg/fqdn/messagehandler/message_handler.go:277 UpdateOnDNSMsg to namemanager/manager.go:214 UpdateGenerateDNS .
4. Name matching. manager.go:422 deriveLabelsForName matches this response name against the regular expressions of all FQDN selectors, and for each matching selector, collects the fqdn:<패턴> labels.
5. Identity to IP. manager.go:315 updateMetadata uploads those labels to the ipcache for each IP in the response, and ipcache assigns an identity matching the label set. The new identity enters pkg/policy/selectorcache.go:768 UpdateIdentities into the selector cache, and an allowance entry is created in the policy map of endpoints using that selector.
6. Wait and return. UpdateGenerateDNS returns a channel that waits until ipcache propagates that change down to the map(WaitForRevision). The handler waits only for ( tofqdns-proxy-response-max-delaysmall cluster 100ms) and writes the response to the pod. This order ensures that the policy map is already open as soon as the pod connects upon receiving the response.

It is important to note here that matchpattern is used twice (③ and ⑤ in the figure). The translation rules are short.

func escapeRegexpCharacters(pattern string) string {
	// Convert '.' in the match pattern as literal '.' for regex pattern.
	pattern = strings.ReplaceAll(pattern, ".", "[.]")

	// '**.' in match pattern prefix is a subdomain wildcard specifier which matches one ore more
	// entire labels.
	pattern = subdomainWildcardSpecifierPrefix.ReplaceAllString(pattern, dnsWildcardREGroup)

	// Base case: * becomes .*, but only for DNS valid characters
	// `*` wildcard matches all DNS characters within the subdomain boundary(doesn't include '.' literal)
	pattern = wildcardSpecifier.ReplaceAllString(pattern, allowedDNSCharsREGroup+"*")

	return pattern
}

Source: cilium v1.20.1 pkg/fqdn/matchpattern/matchpattern.go lines 131–144, Apache-2.0

* is changed to [-a-zA-Z0-9_]*(line 17), and cannot cross dots. Receiving multiple segments is only done by the very first **. (in line 37, ^[*]{2,}\[\.\], in line 23 dnsWildcardREGroup), and the middle ** simply turns into * on the last line. Normalization appending a dot at the end is handled by Sanitize(line 87) .

Measurement case. This is a record measured in a small cluster while preparing a contribution (2026-10-16~17, PR draft). The policy for the image registry is toFQDNs: matchPattern: "*.docker.com" and DNS is * accessed each of the three names twice from the opened pod.

Since the second line is the CDN name that downloads Docker Hub's image layers, the policy seems correct, but it results in a symptom where only image pulling fails. In the same logs, the result of directly running main's matchpattern in Go(Validate(Sanitize(p))) was also identical to the table(**.docker.com matches the second line), and the result of converting the conversion rules to Python and running them again while writing this post was also the same (table below the figure). Since all DNS queries were resolved, the blockage is at ⑤, not ③. How the same characteristic caused an issue in the practice pod's DNS egress policy is described separately in Blocking Leaks via DNS . This post only covers which line of code produces that characteristic.

4.4 Hubble: From perf ring buffer to relay

1. BPF generates events. When a packet is forwarded, dropped, or evaluated by a policy, bpf/lib/trace.h:210 _send_trace_notify and others write short records using cilium_events(bpf/lib/events.h:13, PERF_EVENT_ARRAY). Writing every packet is expensive, so it is reduced using MONITOR_AGGREGATION . The measured endpoint's ep_config.h was MONITOR_AGGREGATION 3(summarizing active connection tracking, ConfigMap's monitor-aggregation: medium).
2. The agent reads them. pkg/monitor/agent/agent.go:324 handleEvents opens the perf reader, and distributes records to consumers per notifyPerfEventLocked(line 428) . Hubble is one of those consumers(pkg/hubble/monitor/consumer.go:73).
3. interpreting into the ring buffer. pkg/hubble/observer/local_observer.go:116 Start 's loop turns events into Decode flows, passes them through metrics hooks, and puts them into a fixed-size ring via container/ring.go:168 Ring.Write . Since the size is a power of 2 minus 1 (default 4095), the index is obtained using a mask instead of division. GetFlows(line 260) only reads this ring.
4. Relay collects them. relay connects to each agent's Hubble server (:4244, TLS) in advance(pkg/hubble/relay/pool/manager.go:51). When a request arrives, pkg/hubble/relay/observer/server.go:62 GetFlows sends the same request to all peers(observer.go:37), and re-sorts the returned flows chronologically(sortFlows, line 67) before outputting them.

The design philosophy is "Do not store." Flows exist only in the node memory ring, and the relay does not retain them either. The measured node had its ring full at 4095/4095 with 58.5 inflows per second, so the past remembered by a single node is about 70 seconds. To retain data longer, use separate metrics or exporters.

5. Connection to User Contributions

The author has prepared changes to correct the characteristics of 4.3 in the documentation (Section 5 of the original ledger, "Prepared Only"). It has not been published yet. Because Cilium has a policy of not accepting communications (such as PR bodies) largely written by AI, the author must write the main text directly. Therefore, even here, we only write up to "preparation completed."

The basis is the actual measurement table in 4.3 and the Go reproduction result. The draft states that it was confirmed during preparation that three document files and matchpattern.go are identical between v1.20.1 and main.

6. Actual Measurements

2026-10-17, sent read commands only to the agent pod on one control plane node in a small cluster (8 nodes, no kube-proxy DaemonSet). Raw data is stored separately; the values below exclude names and addresses.

There are three things to read.

• Socket LB is only half enabled. This is the ENABLE_SOCKET_LB_HOST_ONLY path seen in 4.2. It is also visible in the number of programs. Each endpoint has tail_nodeport_rev_dnat_ipv4·tail_no_service_ipv4 similar tc-side LB tail calls attached, and the socket reverse map has only 13 slots.
• There is no pod-side egress hook. to the pod Incoming packets are not handed over from the egress of veth, but directly after the host-side program invokes a policy tail call (cil_lxc_policy). This is the case where condition 4.1 in RequireEgressProg() is false. This differs from the common diagram depicting "two programs per pod".
• No FQDNs were observed on this node. Since only one agent was read, the cache of the node with pods subject to DNS rules is unmeasured.

7. Where to Read From

1. daemon/cmd/cells.go:103 — Browsing the cell list to map what the agent contains. The comments are helpful.
2. pkg/loadbalancer/README.md and bpf_reconciler.go:842 — The shortest place to learn Cilium's new control plane syntax of "table → reconciler → map".
3. bpf/bpf_sock.c:298 — The side that reads maps. Let's see how the slot used in #2 is read.
4. pkg/datapath/loader/endpoint.go:195 — Template, constant, and tcx attachment order.
5. pkg/fqdn/dnsproxy/proxy.go:895 to namemanager/manager.go:214 — Where the userspace proxy and BPF policies meet.
6. bpf/bpf_lxc.c:1780 cil_from_container — Read last. There are many tail calls and #ifdef , so you need to understand the first five to avoid getting lost.

8. Good First Issues

These are actual issues encountered while reading the source code (not published, and it has not been verified whether they remain as-is in main).

• pkg/loadbalancer/README.md:33 Filename without . Although the Kubernetes data source is said to be in reflector.go , that file does not exist in v1.20.1 and the actual implementation is reflectors/k8s.go (the list on line 61 of the same README correctly points to the reflectors directory). This is a good candidate for a one-line documentation fix.
• bpf/bpf_lxc.c:88 Typo in comments. In 「enable per-packet LB is SCTP is enabled」, 「is」 should be 「if」. This is a line where readers pause, as it sits right above the condition cited in 4.2.
• pkg/fqdn/matchpattern/matchpattern.go:19·:135 「one ore more」 in . Two typos. Also, the documentation comment in ToAnchoredRegexp(lines 95–99) explains only * and does not mention **. . Since this is the same topic as the documentation change in Section 5, it is good to handle them together.
• Explanation of the wildcard header cell. One-third of the measured header cells were port 0/ANY cells, and their usage could not be found except for a single-line bpf-lb-enable-wildcard-entries option comment. bpf lb list This is a documentation candidate explaining /ANY (whether prior documentation exists has not been verified).
• Official guide. If you are a beginner, look at the contributing guide and start with the good-first-issue label it points to. AI Policy: Cilium does not accept PR bodies and comments heavily generated by AI, so communications must be written yourself.

9. Limitations

• What was not covered: ClusterMesh, BGP control plane, WireGuard/IPsec encryption, Gateway API/Ingress Envoy configuration paths (pkg/ciliumenvoyconfig), egress gateway, Maglev, XDP acceleration, IPv6, identity allocation details, and operator IPAM allocator (included via build tag ipam_provider_* — small clusters do not use ipam: kubernetes ).
• Unmeasured: FQDN cache and selectors on nodes with pods subject to DNS rules, cases where responses are sent earlier by bypassing tofqdns-proxy-response-max-delay , BPF compilation time (templates were created about 6 weeks prior to agent startup and could not be observed this time), and endpoint recovery upon node restart.
• The connection table in 4.3 was not remeasured this time (2026-10-17) but recorded during contribution preparation, due to this measurement's rule of not spawning new pods.
• Envoy only documents xDS relationships. L7 policy paths were unmeasured (Proxy Status showed 「0 redirects active」).

A word from TT. Cilium's checkpoint doesn't pick up the phone. It only looks at the rule sheet it received in advance. So if something is weird, instead of the checkpoint, go find who wrote that sheet. And also, remember that * is a coward who cannot cross even a single dot.

Note

All materials were accessed on 2026-10-17. The source is the cilium/cilium v1.20.1 tag (commit 7d68cfb394f2).

• Agent Cell: https://github.com/cilium/cilium/blob/v1.20.1/daemon/cmd/cells.go, https://github.com/cilium/cilium/blob/v1.20.1/pkg/datapath/cells.go
• BPF Loading: https://github.com/cilium/cilium/tree/v1.20.1/pkg/datapath/loader, https://github.com/cilium/cilium/blob/v1.20.1/pkg/datapath/orchestrator/orchestrator.go
• BPF Program: https://github.com/cilium/cilium/tree/v1.20.1/bpf (bpf_lxc.c, bpf_host.c, bpf_overlay.c, bpf_sock.c, bpf_xdp.c, lib/lb.h, lib/trace.h, lib/events.h)
• Load Balancing Control Plane: https://github.com/cilium/cilium/tree/v1.20.1/pkg/loadbalancer
• FQDN: https://github.com/cilium/cilium/tree/v1.20.1/pkg/fqdn, https://github.com/cilium/cilium/blob/v1.20.1/pkg/policy/api/fqdn.go, https://github.com/cilium/cilium/blob/v1.20.1/pkg/policy/selectorcache.go
• Hubble: https://github.com/cilium/cilium/tree/v1.20.1/pkg/hubble, https://github.com/cilium/cilium/tree/v1.20.1/pkg/monitor/agent, https://github.com/cilium/cilium/tree/v1.20.1/hubble-relay
• operator: https://github.com/cilium/cilium/blob/v1.20.1/operator/cmd/root.go, https://github.com/cilium/cilium/tree/v1.20.1/operator/pkg/ipam
• kube-proxy replacement documentation (socket LB bypass): https://github.com/cilium/cilium/blob/v1.20.1/Documentation/network/kubernetes/kubeproxy-free.rst
• DNS policy documentation: https://github.com/cilium/cilium/blob/v1.20.1/Documentation/security/policy/layer3.rst, https://github.com/cilium/cilium/blob/v1.20.1/Documentation/security/policy/layer7.rst
• Contributing guide: https://github.com/cilium/cilium/blob/v1.20.1/Documentation/contributing/development/contributing_guide.rst
• Other parts in the same series: Reading MetalLB(Who advertises the address of a LoadBalancer service)

Cited code: bpf/bpf_sock.c·bpf/bpf_lxc.c is GPL-2.0-only OR BSD-2-Clause, pkg/fqdn/matchpattern/matchpattern.go is Apache-2.0 (Copyright Authors of Cilium).