Writing a CDI Specification
Goal
Write a CDI (Container Device Interface) spec yourself, verify its syntax and placement, and write the format of the podman run --device argument correctly.
Why it matters
CDI is a vendor-neutral standard that describes "which device nodes, libraries, and environment variables are needed to put this device into a container." It is more transparent and more portable than the older runtime-hook approach, and podman, containerd, and CRI-O all support it. In practice nvidia-ctk cdi generate creates it automatically, but because the cause of "the GPU is not detected after a driver update" is mostly a missed spec regeneration, you need to be able to read its contents.
There is no real GPU in this environment. So grading covers the spec's syntax, structure, and placement, and the format of the --device argument — those are exactly the places where mistakes happen in the field.
Steps
- Create the
/etc/cdidirectory. Also create the/root/cdiworking directory. - Create
/etc/cdi/nvidia.yaml. At the top level it must havecdiVersion: "0.6.0"andkind: nvidia.com/gpu, and thedevicesarray must have one entry withname: "0". In that entry,containerEdits.deviceNodesmust contain the three paths/dev/nvidia0,/dev/nvidiactl, and/dev/nvidia-uvm. - Parse that file as YAML and save the list of top-level keys to
/root/cdi/parsed.txt. All three keyscdiVersion,kind, anddevicesmust be visible. - Add a second entry with
name: "all"to thedevicesarray. ItsdeviceNodesmust also have at least two paths. - Add
mountsto thecontainerEditsof thename: "0"entry. BothhostPathandcontainerPathmust be/usr/lib/x86_64-linux-gnu/libnvidia-ml.so.550.90.07, andoptionsmust contain the four valuesro,nosuid,nodev, andbind. - Add
hooksto thecontainerEditsof thename: "0"entry.hookNameiscreateContainer,pathis/usr/bin/nvidia-ctk, andargsis["nvidia-ctk", "hook", "update-ldcache"]. - Write
/root/cdi/run.sh. It must contain a command that requests thenvidia.com/gpu=alldevice withpodman run. The format of the--deviceargument value must be exact. - Create
/root/cdi/report.txtwith the following 5 lines. The values must come from parsing the spec you wrote.CDI_VERSION=0.6.0/KIND=nvidia.com/gpu/DEVICES=<devices 배열 길이>/NODES_DEV0=<name 이 "0" 인 장치의 deviceNodes 개수>/SPEC_PATH=/etc/cdi/nvidia.yaml(the placeholders are the length of the devices array and the number of deviceNodes of the device whose name is "0")
Notes
- Parse YAML in a form like
python3 -c "import yaml,sys;d=yaml.safe_load(open('/etc/cdi/nvidia.yaml'));print(list(d))". - The
--devicevalue has the form<kind>=<장치이름>(the placeholders are the kind and the device name). - In a real environment,
nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yamlcreates this file for you. - Common mistake 1: writing
kindlikenvidia/gpu. The vendor part must be in domain form (nvidia.com). - Common mistake 2: writing the device name
"0"without quotes so that YAML parses it as a number. It must be a string.
Prepare the CDI directory
Create the /etc/cdi directory. Also create the /root/cdi working directory.
There are two standard paths. Use the one where persistent specs go.
Write a minimal spec
Create /etc/cdi/nvidia.yaml. At the top level it must have cdiVersion: "0.6.0" and kind: nvidia.com/gpu, and the devices array must have one entry with name: "0". In that entry, containerEdits.deviceNodes must contain the three paths /dev/nvidia0, /dev/nvidiactl, and /dev/nvidia-uvm.
You need two keys at the top level and the devices array. kind joins a vendor in domain form and a class with a slash.
Verify the YAML syntax
Parse that file as YAML and save the list of top-level keys to /root/cdi/parsed.txt. All three keys cdiVersion, kind, and devices must be visible.
If you read it with python3's yaml module, syntax errors show up immediately. Summarize the parse result and save it.
Add a second device
Add a second entry with name: "all" to the devices array. Its deviceNodes must also have at least two paths.
By convention, all is the name that means every device. Put one more entry in the devices array.
Add a library mount
Add mounts to the containerEdits of the name: "0" entry. Both hostPath and containerPath must be /usr/lib/x86_64-linux-gnu/libnvidia-ml.so.550.90.07, and options must contain the four values ro, nosuid, nodev, and bind.
A mounts entry has three keys: hostPath, containerPath, and options. options is an array of strings.
Add a hook
Add hooks to the containerEdits of the name: "0" entry. hookName is createContainer, path is /usr/bin/nvidia-ctk, and args is ["nvidia-ctk", "hook", "update-ldcache"].
A hooks entry holds hookName, path, and args. Use the name of the hook that runs when the container is created.
Write the run command
Write /root/cdi/run.sh. It must contain a command that requests the nvidia.com/gpu=all device with podman run. The format of the --device argument value must be exact.
The value of --device has the form kind and device name joined with an equals sign. Keep it as a script.
Spec verification report
Create /root/cdi/report.txt with the following 5 lines. The values must come from parsing the spec you wrote.
CDI_VERSION=0.6.0 / KIND=nvidia.com/gpu / DEVICES=<devices 배열 길이> / NODES_DEV0=<name 이 "0" 인 장치의 deviceNodes 개수> / SPEC_PATH=/etc/cdi/nvidia.yaml (the placeholders are the length of the devices array and the number of deviceNodes of the device whose name is "0")
The values must come from parsing the spec you actually wrote. The counts are array lengths.