TT Lab
Blog

한국어 English 日本語 中文

Get started
Blog

Blog

Reading KubeVirt Source Code 3: virt-controller — Reconciling VMs and VMIs

Status: Draft (2026-10-17), Source + Read-only measurements. Based on KubeVirt v1.9.0(commit 79d34c17). Measurements read logs, objects, and pod specs from a small cluster running v1.9.0, without creating or modifying any VMs.

Reading KubeVirt Source Code 3: virt-controller

Table of Contents

1. Intuition: Only names in the queue, judgment always starts from scratch

Kubernetes controllers operate based on "what the current state is" rather than "what happened." The event handler only puts the name(네임스페이스/이름) of the changed object into the work queue, and a worker takes that name, reads the latest object from the cache, and recalculates the "difference between the desired state and the current state" from scratch. Even if the same name enters ten times, it is processed only once, and if it fails, the name is delayed and re-queued.

A single VM in virt-controller goes through these two loops.

2. Diagram

virt-controller — 두 컨트롤러가 이어 달린다 VM 컨트롤러가 VMI 를 만들고, VMI 컨트롤러가 파드를 만든다 VM 컨트롤러 pkg/virt-controller/watch/vm · 일꾼 3 인포머 VM·VMI·DV·PVC 작업 큐 키 = ns/이름 Execute() 실패 → 재시도 sync() — 종료자 · DataVolume 준비 · 리비전 syncRunStrategy() — Always · Halted · Manual … startVMI() → VMI 생성 stopVMI() → VMI 삭제 VMI 컨트롤러 watch/vmi · 일꾼 10 sync() 파드가 없으면 만든다 RenderLaunchManifest() services/template.go createPod() → virt-launcher 파드 updateStatus(): Pending → Scheduling → Scheduled 파드가 Ready 면 노드 이름을 적고 virt-handler 에 넘긴다 실측: 리스 virt-controller 는 한 파드가 쥐고, 다른 하나는 대기한다. 실측: Failed 로 끝난 VMI 를 Always 가 지우고(vm.go:1005) 0.19초 뒤 새 VMI 를 만들었다(vm.go:1341). 로그의 줄 번호가 v1.9.0 소스와 같다.
Figure: The relay-style reconciliation between the VM controller and the VMI controller

3. Code path: Must become the leader to work

Even if two pods are running, virt-controller runs the controller on only one leader . In application.go 's leader election callback onStartedLeading()(:609), it starts informers and launches the controller like go vca.vmController.Run(...) . The default number of workers is 10 for the VMI controller (:128), and 3 for most others including the VM controller (:125).

4. VM Controller: pkg/virt-controller/watch/vm/vm.go

4.1 Execute — Dequeue an item

vm.go:336 of the Execute() has the same shape across all controllers.

if err := c.execute(key); err != nil {
	log.Log.Reason(err).Infof("re-enqueuing VirtualMachine %v", key)
	c.Queue.AddRateLimited(key)
} else {
	log.Log.V(4).Infof("processed VirtualMachine %v", key)
	c.Queue.Forget(key)
}

Source: kubevirt/kubevirt v1.9.0 vm.go lines 347–353 (Apache-2.0). Failures are retried with an increasing AddRateLimited , and upon success, Forget clears the backoff history.

4.2 execute → sync

execute()(:357) reads the VM from the cache (indexer), makes a copy, finds a VMI with the same name, checks the ownership (ownerReference) (}, {ClaimVirtualMachineInstanceByName), gathers the list of DataVolumes, and calls sync() and updateStatus() . Directly modifying objects in the cache would pollute the cache seen by other workers, so KubeVirt code almost always starts with DeepCopy() . This is a frequent comment in code reviews.

sync()(:3215) is structured as follows.

  1. Check expectations. satisfiedExpectations() If is false, it returns without doing anything. This is because a request to create a VMI was just made, but that VMI is not yet visible in the informer. Without this mechanism, a lagging cache would cause the VMI to be created twice.
  2. If it is being deleted, stops the VMI and removes the finalizer. Otherwise, it adds the finalizer kubevirt.io/virtualMachineControllerFinalize .
  3. vm.RunStrategy()(Part 2) folds the run strategy into a single option, synchronizes the instancetype, and verifies DataVolume creation and preparation.
  4. syncRunStrategy() creates or deletes the VMI using

4.3 syncRunStrategy — Always branch

vm.go:970 's switch runStrategy is the heart of this controller. The Always(:979) branch can be described in plain words as follows.

RerunOnFailure reflects the difference that Halted recreates only when Failed and leaves Succeeded as is, while Manual follows only requests injected by subresources (Part 2), as written in each branch.

4.4 startVMI — Things done before creating a VMI

startVMI()(:1257) does not immediately create a VMI, but instead performs five steps first.

  1. SetupVMIFromVM()(:1868— Copies the VM template, populates the name, namespace, and owner (VM), and if the template lacks a firmware UUID, calculates and inserts one from the VM name (CalculateLegacyUUID). This is why the UUID seen by the guest remains the same even after restarting the VM.
  2. createVMRevision() — Saves the VM spec used for this startup as ControllerRevision . The name is revision-start-vm-<VM UID>-<generation> (getVMRevisionName, :1698). Later, when the VM spec changes, this serves as the baseline for calculating the "RestartRequired" condition.
  3. Attaches a finalizer to the VMI. This ensures that the VM controller is guaranteed to inspect the VMI at least once before it is deleted.
  4. Applies the instancetype and default network interface, and re-validates the network.
  5. c.expectations.ExpectCreations(vmKey, 1)(:1333) to record "create one" and creates the VMI. When the informer sees the new VMI, addVirtualMachineInstance() lowers its expected value to CreationObserved .

5. VMI Controller: Creating and Handing Over the Pod

5.1 sync in lifecycle.go

The VMI controller's decision-making logic is located in pkg/virt-controller/watch/vmi/lifecycle.go 's sync() (:66). If the VMI is being deleted, it deletes the pod; if deletion is complete, it cleans up only the hotplug pods. If no pod exists yet and the VMI is being processed for the first time, it checks several conditions (DataVolume readiness, no ongoing migration, backend storage readiness), draws the pod using RenderLaunchManifest() (:156), and creates it using createPod() (:1076). If rejected due to Pod Security Standards, it translates the error into a user-friendly message stating that a "privileged namespace is required" and records it as an event.

5.2 updateStatus — The Handover Moment

updateStatus()(:256) transitions the phase. Once the pod is created, Pending → Scheduling, and in Scheduling , once the pod becomes Ready , it populates the volume and network statuses, changes the phase to Scheduled , and applies the labels kubevirt.io/nodeName=<노드> and status.nodeName to the VMI (:421–426). These labels are precisely what virt-handler uses in Part 4 to select its workload. As seen in the comment from Part 2 in Scheduled , this is where the baton is passed.

6. Pod Template: pkg/virt-controller/services/template.go

template.go 가 그린 virt-launcher 파드 실측: 2 vCPU · 2Gi VM 하나(PVC 루트) — 괄호는 컨테이너 디스크 VM 에만 Pod virt-launcher-<vm>-xxxxx · restartPolicy Never terminationGracePeriodSeconds 60 = 30 + 15 + 15 initContainers guest-console-log (restartPolicy: Always) 네이티브 사이드카 · virt-tail · CPU 5m · 메모리 35M (container-disk-binary · volumeroot-init) 컨테이너 디스크를 미리 받아 두는 초기화 containers compute virt-launcher-monitor --name … --grace-period-seconds 45 요청: CPU 200m (= 2 vCPU ÷ 10) · 메모리 2328Mi = 게스트 2048Mi + 오버헤드 280Mi 장치: devices.kubevirt.io/kvm · tun · vhost-net (volumeroot — 디스크 이미지를 품은 컨테이너) volumes private public sockets libvirt-runtime ephemeral-disks container-disks hotplug-disks root ← PVC 오버헤드 280Mi 는 코드의 추정식과 정확히 같다: 고정 220Mi + 페이지 테이블 4Mi(2Gi ÷ 512) + vCPU 8Mi×2 + IO 스레드 8Mi + 비디오 32Mi (hypervisor/kvm GetMemoryOverhead) 8 vCPU · 8Gi 실습 VM 은 340Mi — 역시 식과 같다
Figure: virt-launcher pod and memory overhead drawn by template.go

RenderLaunchManifest()(:355) first calculates the memory overhead, and delegates the actual assembly to renderLaunchManifest() (:395). There are five key areas to look at in this nearly 400-line function.

7. Measurements

Verified through inspection on a small cluster running v1.9.0 (2026-10-17).

Leader. The holder of the lease kubevirt/virt-controller was one of the two pods (lease duration 15serializers/seconds), while the other pod's logs stopped at a single line during startup: "attempting to acquire leader lease". The standby pod only targets the lease and runs no controllers.

Log of a Single Restart. On 2026-10-13, a VMI for a VM (runStrategy: Always) ended in Failed (the reason for termination is out of scope for this article). Extracting only the lines for that VM from the leader's logs clearly reveals the branching logic from Section 4.3. All line numbers for pos match the v1.9.0 source code.

Time (UTC) pos Message
00:42:32.011 vm.go:1005 Stopping VM with VMI in phase Failed and VM runStrategy: Always
00:42:32.042 vm.go:1685 Dispatching delete event for vmi … with phase Failed
00:42:32.179 vm.go:1032 Starting VM due to runStrategy: Always
00:42:32.196 vm.go:1341 Started VM by creating the new virtual machine instance …

It took 0.19 seconds from the initial judgment to the creation of the new VMI. It is also visible that deletion and creation occurred in different reconciles (with a Stopping line for the same VM in between).

Traces Left on the New VMI. That VMI's status.virtualMachineRevisionName was revision-start-vm-<VM UID>-1 (Section 4.4, item 2), and metadata.finalizers and kubevirt.io/virtualMachineControllerFinalize , kubevirt.io/foregroundDeleteVirtualMachine, annotations kubevirt.io/vm-generation: "1", and labels kubevirt.io/nodeName were attached. Phase history (phaseTransitionTimestamps) was Pending·Scheduling 00:42:32 → Scheduled 00:42:35 → Running 00:42:36.

Pod specs and formulas. We read a virt-launcher pod for the same VMI (2 vCPU, 2Gi guest, root on PVC) and a practical VM (8 vCPU, 8Gi, container disk).

Item 2 vCPU · 2Gi 8 vCPU · 8Gi Code
CPU Request 200m 800m vCPU ÷ 10
Memory Request 2328Mi 8532Mi Guest + Overhead
Overhead (Annotation kubevirt.io/memory-overhead-bytes) 293,601,280 B = 280Mi 356,515,840 B = 340Mi Calculation below
--grace-period-seconds / Pod Grace Period 45 / 60 45 / 60 30+15, +15
--qemu-timeout 283s 309s 240 + 0–119
User runAsUser 107, runAsNonRoot Same Non-root default

Recalculating the overhead formula, a 2Gi VM has fixed 220Mi + page table 4Mi + vCPU 16Mi + IO thread 8Mi + graphics 32Mi = 280Mi, and an 8Gi VM has 220 + 16 + 64 + 8 + 32 = 340Mi , which matches the measured annotations down to the byte. This is where the "about 280Mi" from part 2 of "Until the Student VM Boots" comes from.

Additionally, the pod had a readiness gate kubevirt.io/virtual-machine-unpaused(pausing the VM removes the pod from Ready), device resource devices.kubevirt.io/kvm·tun·vhost-net requests, and guest-console-log init container(restartPolicy: Always, CPU 5m, Memory 35M).

8. Limitations

A word from Titi. When reading controller code, constantly ask, "What happens if this function fails?" The answer is almost always, "Put the name back in the queue and look at it from the beginning next time." That is why even if a function is slightly flawed, it eventually converges to the correct state!

References

All accessed on 2026-10-17.

Sign in to like

Comments

No comments yet.

Sign in to leave a comment