Status: Draft (2026-10-16), official documentation + partial measurements. KubeVirt, CDI, and Kubernetes documentation was checked on 2026-10-16 (KubeVirt v1.9.0, CDI v1.66.0, Kubernetes v1.34). Storage class settings were queried as read-only on the same day, and as this is a public post, storage product and driver names have been generalized to "shared storage CSI driver" and "node-local class." Writing rule: Measured data is the default; unmeasured items are marked as "Unmeasured," and items not found in documentation are marked as "Unverified."
Until Student VMs Spin Up 3: Disks Are PVCs
Table of Contents
- 1. Types of Volumes
- 2. Container Disk: Fast Yet Small
- 3. DataVolume: Large Yet Slow
- 4. Access Modes: How Many Nodes Can Attach Simultaneously
- 5. Volume Modes: Filesystem or Block
- 6. Binding Timing and StorageProfile
- 7. Summary: Three Paths of Trade-offs
- References
Part 1, item ⑦ "Attaching Volumes." To put the conclusion first, a KubeVirt disk is a pair consisting of **a device visible to the guest (disk)** and **a Kubernetes volume containing its contents (volume)**. In the manifest, the two are linked by name.
domain:
devices:
disks:
- name: root # 게스트에게는 virtio 디스크(/dev/vda)
disk: {bus: virtio}
volumes:
- name: root # 내용은 이 볼륨에서 온다
containerDisk: {image: quay.io/containerdisks/ubuntu:24.04}
Depending on what you put in the volume slot, the characteristics of the disk change completely.
1. Types of Volumes
Among the volume sources highlighted by the KubeVirt User Guide, five are used in the lab VMs.
| Source | What It Is | When Turned Off | In Labs |
|---|---|---|---|
containerDisk |
Disk image inside a /disk container image |
Disappears (ephemeral) | Default root disk |
emptyDisk |
Empty sparse qcow2 created on the node. Capacity must be specified | Persists across guest reboots, disappears when the VM is recreated | Scratch space for Docker and Kubernetes labs |
persistentVolumeClaim |
Pre-existing PVC | Persists | — |
dataVolume |
PVC created by CDI by fetching an image | Persists (deleted along with the VM if owned by the VM) | Persistent disk option |
ephemeral |
PVC used as a read-only basewith writes going to a local write layer | Only the write layer disappears | Golden disk (Part 4) |
cloudInitNoCloud |
cloud-init data (from a Secret) | — | First boot configuration (Part 1 ③) |
2. Container Disk: Fast Yet Small
A container disk receives the OS image from a registry just like a container image. Once a node pulls it once, subsequent VMs use it immediately without pulling again. Because the disk is ephemeral, it fits labs that require "create, break, and delete," making it the default. As measured in Part 1, it reaches Running in around 20 seconds.
Being ephemeral means that the base is an image file cached on the node, and guest writes accumulate separately. Inspecting inside virt-launcher of an operator VM that had been running for 9 days, the root was a qcow2 write layer (allocated ~1.10GiB) layered on top of the base (disk image cached on the node, allocated ~596MiB) (measured, 2026-10-14). The golden disk covered in Part 4 is this structure with only the base changed to a PVC.
The drawback is size. The root of a container disk is only as large as the virtual size of the base image—just 2.4GiB for the Ubuntu image we use (measured). We hit roadblocks twice because of this while building the labs.
- Docker labs. After pulling a few images and building a couple of times, the root filled up. However, all errors were of the "image not found" variety, making it hard to notice that space was the cause. We attached a 20GiB
emptyDiskas scratch and moved/var/lib/docker, but it filled up again. Docker 29 uses containerd's image store, so layers accumulated on the/var/lib/containerdside. The issue was only resolved after moving both paths. - Scratch disk naming. We named the scratch
/dev/vdband tried to format it, but the cloud-init disk took that slot, making the scratch/dev/vdcinstead (measured). Getting the name wrong formats the cloud-init disk. Therefore, we select it by its properties (no filesystem, no partitions, and a disk larger than 5GB) instead of its name. - Kubeadm labs. While pulling packages and control plane images, it failed with
No space left on device. Since there were many paths difficult to move to scratch, the root itself needed to be large.
3. DataVolume: Large Yet Slow
For labs requiring a large root, the persistent option is available. Here, the backend puts dataVolumeTemplates in the VM manifest, and CDI (Containerized Data Importer) creates a PVC for each VM to fetch the image.
The sequence for CDI to fetch a registry image is as follows (CDI "Scratch space" documentation): the importer pod receives the image, unpacks the layers, locates the disk image, converts it to qemu-img , and writes it to the target PVC. This process temporarily requires a scratch PVC of the same size as the target , which is returned when finished. Consequently, while fetching a single 10Gi disk, two PVCs are created.
Because the VM owns the DataVolume, deleting the VM deletes the PVC as well. The issue is time. Lab sessions are structured to create a new VM every time, so fetching a fresh image every time a VM is created pushed the first boot time up to 195 seconds max(measured, recording maximum values during lab development). This is also why separate upper limits for backend VM preparation waiting times were set for container disks (150 seconds) and DataVolumes (300 seconds).
There is also a catch with size. In filesystem PVCs, KubeVirt places the disk as a raw file named disk.img at the PVC root, so space is taken up by whatever the filesystem itself occupies. According to CDI documentation, only when a DataVolume is written as spec.pvc instead of spec.storage , CDI calculates this share (filesystem overhead) and requests a larger PVC. Since the overhead in TT Lab's CDI configuration is 6% (verified 2026-10-16), requesting 10Gi as spec.storage will result in a guest disk of 10Gi and a slightly larger PVC (calculated, unmeasured). The design document stated that "the 10Gi golden guest root will be about 9.4GiB," which contradicts the CDI documentation. Which side is correct will be confirmed by measurements at the end of Part 4. Additionally, KubeVirt accepts disk.img even if it is slightly smaller than requested (up to a default of 10%) (User Guide).
4. Access Modes: How Many Nodes Can Attach Simultaneously
PVC access modes are particularly important for VMs. Summarizing the Kubernetes documentation definitions:
| Mode | Meaning | In VMs |
|---|---|---|
| ReadWriteOnce (RWO) | Attached as read-write bya single node | . Multiple pods on the same node can share it |
| The VM is bound to that node. Live migration is impossible | ReadOnlyMany (ROX) | Attached as read-only by multiple nodes |
| Seems appropriate for a base disk, but see caution below | ReadWriteMany (RWX) | Attached as read-write by multiple nodes |
| Live migration possible, multiple nodes share the same base | ReadWriteOncePod (RWOP) | Only a single pod across the entire cluster |
CSI volumes only
Two points to keep in mind. Access modes are not write protection. Kubernetes documentation clearly states that access modes do not prevent writes after attachment, and volumes created with ROX are not guaranteed to be read-only. True protection ismounting the volume as read-only readOnly: true . This is why Part 4's golden disk is created as RWX but does not omit
when attached. Live migration requires RWX. LiveMigratable The KubeVirt User Guide states that VMs using PVCs can only be migrated when the access mode is RWX. Whether migration is possible is calculated when the VMI starts and is left as a LiveMigration condition, and methods are divided into BlockMigration (migrating only memory) and LiveMigratable=True(which must also copy the disk). In fact, inspecting a VM using a container disk and emptyDisk, the method was BlockMigration (queried 2026-10-16). This means node-local disks must be entirely copied to be migrated. TT Lab's DataVolume option is created as RWO, so that VM cannot be migrated. Since lab VMs do not need to be migrated anyway, this is an accepted trade-off.
5. Volume Modes: Filesystem or Block
| Mode | Shape Visible to Pod | KubeVirt Disk |
|---|---|---|
| Filesystem (default) | Mounted as a directory | A disk.img(raw) file inside it. Owner must be UID 107 |
| Block | Block device without a filesystem | The device used directly as a disk |
Kubernetes documentation describes block mode as the fastest path with no filesystem layer between the pod and volume. Instead, the consumer must know how to handle block devices, which QEMU does. TT Lab's shared storage class reports that CDI only provides filesystem mode (Section 6 below). Therefore, the golden disk is also disk.img inside a filesystem PVC.
6. Binding Timing and StorageProfile
Whether a volume is bound immediately upon PVC creation (Immediate) or delayed until a pod using that PVC is scheduled on a node (WaitForFirstConsumer) also creates differences for VMs. Node-local classes must use the delayed approach. Because the volume is a disk on a specific node, binding before knowing where the pod will land would cause it to be created on the wrong node.
CDI provides a feature gate (HonorWaitForFirstConsumer) that respects this delay. Consequently, fetching is also delayed until the first consumer (VM) lands. Volumes that must be fully fetched before the VM arrives , like golden disks, are therefore annotated withcdi.kubevirt.io/storage.bind.immediate.requested("bind immediately").
CDI creates a StorageProfile for each storage class, populating it with the appropriate access mode, volume mode, and cloning strategy for that class. Known provisioners have recommended values inserted, while unknown provisioners are left empty. Creating a DataVolume without an access mode on an empty class will fail (CDI StorageProfile documentation). Checking TT Lab's classes on the day of writing showed the following.
| Class | Binding | Expansion | Provided by StorageProfile | Cloning Strategy |
|---|---|---|---|---|
| Shared Storage (CSI Driver) | Immediate | Possible | RWX · Filesystem | copy |
| Node-local | WaitForFirstConsumer | — | Empty | copy |
Only shared storage classes provision volumes that can attach to multiple nodes. Therefore, golden disks are created with this class. The fact that there is no replication strategy copy and snapshot class serves as the rationale for "Why a Write Layer Instead of PVC Cloning" in Part 4.
7. Conclusion: Three-Way Trade
| Container Disk | DataVolume | Golden + Write Layer (Part 4) | |
|---|---|---|---|
| Root Size | 2.4 GiB (Measured) | Desired Size | Golden Size (10Gi/20Gi menus) |
| Action When Creating VM | Almost none if image exists on node | Fetch new image and write to PVC | One empty write layer |
| First Boot | About 20 seconds (Measured) | Up to 195 seconds (Measured) | Not measured |
| PVCs Created per VM | None | One (+ one scratch space during fetch) | None |
| When Turned Off | Write layer disappears | PVC remains (deleted together when VM is deleted) | Write layer disappears |
The golden disk in Part 4 transforms "fetch a new image for every VM" from the middle column into "fetch once and share among VMs".
A word from TT. Access modes define "how many places can attach" rather than "preventing write access." If you want to protect the base disk, make sure to attach it as read-only, whether using RWX or ROX. And since disk full error messages can be misleading, if something seems off, check
df -hfirst!
References
All documents were accessed on 2026-10-16.
- KubeVirt User Guide, Disks and Volumes (Volume Sources,
disk.img·UID 107,pvcTolerateLessSpaceUpToPercent, ephemeral, emptyDisk): https://kubevirt.io/user-guide/storage/disks_and_volumes/ - KubeVirt User Guide, Live Migration (Requires RWX,
LiveMigratable,BlockMigration): https://kubevirt.io/user-guide/compute/live_migration/ - Kubernetes Documentation, Persistent Volumes (Access Modes, "Access Modes Are Not Write Protection", Volume Mode): https://kubernetes.io/docs/concepts/storage/persistent-volumes/
- CDI v1.66.0, Scratch Space: https://github.com/kubevirt/containerized-data-importer/blob/v1.66.0/doc/scratch-space.md
- CDI v1.66.0, StorageProfile: https://github.com/kubevirt/containerized-data-importer/blob/v1.66.0/doc/storageprofile.md
- CDI v1.66.0, DataVolumes(
spec.storageand Filesystem Overhead): https://github.com/kubevirt/containerized-data-importer/blob/v1.66.0/doc/datavolumes.md - TT Lab Code and Logs:
backend/app/vms.py(DISK_SIZE·SCRATCH_SIZE·DOCKER_SCRATCH_PATHSComments,cloud_initScratch Selection),backend/app/main.py(VM_READY_SEC),docs/plans/2026-08-29-vm-labs-design.md,docs/plans/2026-10-14-vm-golden-overlay.md