TT Lab
Blog

한국어 English 日本語 中文

Get started
Blog

Blog

Until Student VMs Spin Up 3: Disks Are PVCs — DataVolume, Access Modes, Volume Modes

Status: Draft (2026-10-16), official documentation + partial measurements. KubeVirt, CDI, and Kubernetes documentation was checked on 2026-10-16 (KubeVirt v1.9.0, CDI v1.66.0, Kubernetes v1.34). Storage class settings were queried as read-only on the same day, and as this is a public post, storage product and driver names have been generalized to "shared storage CSI driver" and "node-local class." Writing rule: Measured data is the default; unmeasured items are marked as "Unmeasured," and items not found in documentation are marked as "Unverified."

Until Student VMs Spin Up 3: Disks Are PVCs

Table of Contents

Part 1, item ⑦ "Attaching Volumes." To put the conclusion first, a KubeVirt disk is a pair consisting of **a device visible to the guest (disk)** and **a Kubernetes volume containing its contents (volume)**. In the manifest, the two are linked by name.

domain:
  devices:
    disks:
      - name: root          # 게스트에게는 virtio 디스크(/dev/vda)
        disk: {bus: virtio}
volumes:
  - name: root              # 내용은 이 볼륨에서 온다
    containerDisk: {image: quay.io/containerdisks/ubuntu:24.04}

Depending on what you put in the volume slot, the characteristics of the disk change completely.

1. Types of Volumes

Among the volume sources highlighted by the KubeVirt User Guide, five are used in the lab VMs.

Source What It Is When Turned Off In Labs
containerDisk Disk image inside a /disk container image Disappears (ephemeral) Default root disk
emptyDisk Empty sparse qcow2 created on the node. Capacity must be specified Persists across guest reboots, disappears when the VM is recreated Scratch space for Docker and Kubernetes labs
persistentVolumeClaim Pre-existing PVC Persists —
dataVolume PVC created by CDI by fetching an image Persists (deleted along with the VM if owned by the VM) Persistent disk option
ephemeral PVC used as a read-only basewith writes going to a local write layer Only the write layer disappears Golden disk (Part 4)
cloudInitNoCloud cloud-init data (from a Secret) — First boot configuration (Part 1 ③)
루트 디스크를 마련하는 세 갈래 A. 컨테이너 디스크 (기본) 레지스트리 이미지 노드가 한 번 받음 노드 캐시 바탕(읽기만) 쓰기 층 emptyDir · VM 마다 루트 2.4GiB · Running 20초(실측) · 끄면 초기화 B. DataVolume (VM 마다 PVC) 레지스트리 이미지 VM 마다 다시 CDI 가져오기 파드 + 스크래치 PVC PVC 하나 RWO · 10Gi 원하는 크기 · 첫 부팅 최대 195초(실측) C. 골든 + 쓰기 층 골든 PVC 운영자가 한 번 · RWX 읽기 전용 readOnly: true 쓰기 층 emptyDir · VM 마다 골든 크기(10·20Gi) · 부팅 시간 미실측 A 와 C 는 같은 원리다: 읽기 전용 바탕 + VM 마다 쓰기 층. 바탕이 노드 캐시냐 PVC 냐만 다르고, 바탕의 크기가 곧 게스트 루트의 크기다.
Figure: Three paths to preparing a root disk

2. Container Disk: Fast Yet Small

A container disk receives the OS image from a registry just like a container image. Once a node pulls it once, subsequent VMs use it immediately without pulling again. Because the disk is ephemeral, it fits labs that require "create, break, and delete," making it the default. As measured in Part 1, it reaches Running in around 20 seconds.

Being ephemeral means that the base is an image file cached on the node, and guest writes accumulate separately. Inspecting inside virt-launcher of an operator VM that had been running for 9 days, the root was a qcow2 write layer (allocated ~1.10GiB) layered on top of the base (disk image cached on the node, allocated ~596MiB) (measured, 2026-10-14). The golden disk covered in Part 4 is this structure with only the base changed to a PVC.

The drawback is size. The root of a container disk is only as large as the virtual size of the base image—just 2.4GiB for the Ubuntu image we use (measured). We hit roadblocks twice because of this while building the labs.

3. DataVolume: Large Yet Slow

For labs requiring a large root, the persistent option is available. Here, the backend puts dataVolumeTemplates in the VM manifest, and CDI (Containerized Data Importer) creates a PVC for each VM to fetch the image.

The sequence for CDI to fetch a registry image is as follows (CDI "Scratch space" documentation): the importer pod receives the image, unpacks the layers, locates the disk image, converts it to qemu-img , and writes it to the target PVC. This process temporarily requires a scratch PVC of the same size as the target , which is returned when finished. Consequently, while fetching a single 10Gi disk, two PVCs are created.

Because the VM owns the DataVolume, deleting the VM deletes the PVC as well. The issue is time. Lab sessions are structured to create a new VM every time, so fetching a fresh image every time a VM is created pushed the first boot time up to 195 seconds max(measured, recording maximum values during lab development). This is also why separate upper limits for backend VM preparation waiting times were set for container disks (150 seconds) and DataVolumes (300 seconds).

There is also a catch with size. In filesystem PVCs, KubeVirt places the disk as a raw file named disk.img at the PVC root, so space is taken up by whatever the filesystem itself occupies. According to CDI documentation, only when a DataVolume is written as spec.pvc instead of spec.storage , CDI calculates this share (filesystem overhead) and requests a larger PVC. Since the overhead in TT Lab's CDI configuration is 6% (verified 2026-10-16), requesting 10Gi as spec.storage will result in a guest disk of 10Gi and a slightly larger PVC (calculated, unmeasured). The design document stated that "the 10Gi golden guest root will be about 9.4GiB," which contradicts the CDI documentation. Which side is correct will be confirmed by measurements at the end of Part 4. Additionally, KubeVirt accepts disk.img even if it is slightly smaller than requested (up to a default of 10%) (User Guide).

4. Access Modes: How Many Nodes Can Attach Simultaneously

PVC access modes are particularly important for VMs. Summarizing the Kubernetes documentation definitions:

Mode Meaning In VMs
ReadWriteOnce (RWO) Attached as read-write bya single node . Multiple pods on the same node can share it
The VM is bound to that node. Live migration is impossible ReadOnlyMany (ROX) Attached as read-only by multiple nodes
Seems appropriate for a base disk, but see caution below ReadWriteMany (RWX) Attached as read-write by multiple nodes
Live migration possible, multiple nodes share the same base ReadWriteOncePod (RWOP) Only a single pod across the entire cluster

CSI volumes only

Two points to keep in mind. Access modes are not write protection. Kubernetes documentation clearly states that access modes do not prevent writes after attachment, and volumes created with ROX are not guaranteed to be read-only. True protection ismounting the volume as read-only readOnly: true . This is why Part 4's golden disk is created as RWX but does not omit

when attached. Live migration requires RWX. LiveMigratable The KubeVirt User Guide states that VMs using PVCs can only be migrated when the access mode is RWX. Whether migration is possible is calculated when the VMI starts and is left as a LiveMigration condition, and methods are divided into BlockMigration (migrating only memory) and LiveMigratable=True(which must also copy the disk). In fact, inspecting a VM using a container disk and emptyDisk, the method was BlockMigration (queried 2026-10-16). This means node-local disks must be entirely copied to be migrated. TT Lab's DataVolume option is created as RWO, so that VM cannot be migrated. Since lab VMs do not need to be migrated anyway, this is an accepted trade-off.

5. Volume Modes: Filesystem or Block

Mode Shape Visible to Pod KubeVirt Disk
Filesystem (default) Mounted as a directory A disk.img(raw) file inside it. Owner must be UID 107
Block Block device without a filesystem The device used directly as a disk

Kubernetes documentation describes block mode as the fastest path with no filesystem layer between the pod and volume. Instead, the consumer must know how to handle block devices, which QEMU does. TT Lab's shared storage class reports that CDI only provides filesystem mode (Section 6 below). Therefore, the golden disk is also disk.img inside a filesystem PVC.

6. Binding Timing and StorageProfile

Whether a volume is bound immediately upon PVC creation (Immediate) or delayed until a pod using that PVC is scheduled on a node (WaitForFirstConsumer) also creates differences for VMs. Node-local classes must use the delayed approach. Because the volume is a disk on a specific node, binding before knowing where the pod will land would cause it to be created on the wrong node.

CDI provides a feature gate (HonorWaitForFirstConsumer) that respects this delay. Consequently, fetching is also delayed until the first consumer (VM) lands. Volumes that must be fully fetched before the VM arrives , like golden disks, are therefore annotated withcdi.kubevirt.io/storage.bind.immediate.requested("bind immediately").

CDI creates a StorageProfile for each storage class, populating it with the appropriate access mode, volume mode, and cloning strategy for that class. Known provisioners have recommended values inserted, while unknown provisioners are left empty. Creating a DataVolume without an access mode on an empty class will fail (CDI StorageProfile documentation). Checking TT Lab's classes on the day of writing showed the following.

Class Binding Expansion Provided by StorageProfile Cloning Strategy
Shared Storage (CSI Driver) Immediate Possible RWX · Filesystem copy
Node-local WaitForFirstConsumer — Empty copy

Only shared storage classes provision volumes that can attach to multiple nodes. Therefore, golden disks are created with this class. The fact that there is no replication strategy copy and snapshot class serves as the rationale for "Why a Write Layer Instead of PVC Cloning" in Part 4.

7. Conclusion: Three-Way Trade

Container Disk DataVolume Golden + Write Layer (Part 4)
Root Size 2.4 GiB (Measured) Desired Size Golden Size (10Gi/20Gi menus)
Action When Creating VM Almost none if image exists on node Fetch new image and write to PVC One empty write layer
First Boot About 20 seconds (Measured) Up to 195 seconds (Measured) Not measured
PVCs Created per VM None One (+ one scratch space during fetch) None
When Turned Off Write layer disappears PVC remains (deleted together when VM is deleted) Write layer disappears

The golden disk in Part 4 transforms "fetch a new image for every VM" from the middle column into "fetch once and share among VMs".

A word from TT. Access modes define "how many places can attach" rather than "preventing write access." If you want to protect the base disk, make sure to attach it as read-only, whether using RWX or ROX. And since disk full error messages can be misleading, if something seems off, check df -h first!

References

All documents were accessed on 2026-10-16.

Sign in to like

Comments

No comments yet.

Sign in to leave a comment