Growing Happens in Two Layers — LVM, fstab, swap, NFS, autofs

In one line

Storage is layers. On top of a block device is LVM, on top of that a filesystem, and on top of that a mount, and each layer knows only the layer below it. So even if you extend the LV, the filesystem does not know; even if you edit fstab, the kernel does not know; and even if you edit /etc/exports, the NFS server does not know. The labs in this module are practice in telling each of those "does not know" things, one at a time, by hand.

Why this exists

The previous module (lfcs-storage) ran in a Pod. There was no mount or pvcreate there, and the reading material itself wrote "we recommend practicing separately in an environment where you can use a virtual machine or loopback devices". The Storage domain of the LFCS is 20%, yet all of it had stayed at conceptual explanation.

This module runs on an Ubuntu 24.04 VM. You are root and systemd is PID 1, so everything works for real. However, there is no empty disk. The scratch disk the platform attaches is already formatted as ext4 and bind-mounted by cloud-init. So the first step is losetup. losetup(8) is the tool that handles loop devices, which make an ordinary file look like a block device, and in practice too, this is exactly how it is used when testing LVM, RAID, and filesystems without disks. Ubuntu cloud images have snap already using a few starting from /dev/loop0, so if you hard-code a number you run into "device is busy". Get a free one with losetup -f --show, and find it again later with losetup -j <파일> (the placeholder is the file).

How it works

LVM has three layers. In the terms of lvm(8), a PV (physical volume) is a block device with an LVM label written on it, a VG (volume group) is a pool that gathers PVs, and an LV (logical volume) is a block device cut out of that pool. A VG manages space in extents (4MiB by default), so lvcreate -L 600M takes 150 extents. Attaching more disks means running pvcreate and then putting them in the VG with vgextend, and you extend the LV after that. Not having to repartition is the reason to use LVM.

Layer	Command to create	Command to extend	Command to view
PV	pvcreate	—	pvs
VG	vgcreate	vgextend	vgs
LV	lvcreate	lvextend	lvs
Filesystem	mkfs.ext4	resize2fs	df

The spot the exam digs at most often is the last two rows. lvextend(8) only grows the block device called the LV, and the ext4 on top of it does not know that it is sitting on a 900MiB device. That is the state where lvs says 900M but df is unchanged. resize2fs(8) matches the filesystem layer, and ext4 can be grown while mounted (online). lvextend -r is the option that handles these two layers at once.

Write fstab with a UUID, not a device name. The first field of fstab(5) is the device, but a name like /dev/sdb1 or /dev/loop3 can change with boot order. The UUID engraved in the filesystem does not change. You read it with blkid, write it as UUID=…, and separate the six fields (device, mount point, type, options, dump, pass) with spaces. After editing, you have to do two more things. systemd translates fstab into .mount units, so run systemctl daemon-reload, and then use mount -a to check right now that the line actually works. If findmnt --verify produces even one [E], you will meet an emergency shell at the next boot.

A swap file must have permissions 600. mkswap(8) issues an "insecure permissions" warning for a file with wide permissions. Swap holds a copy of memory, so it must not be readable by other users. The order is create, chmod 600, mkswap, swapon, and in fstab the mount point position is none and the type is swap.

The NFS server does not know you edited the file. The format of exports(5) is 디렉터리 클라이언트(옵션) (directory, then client, then options in parentheses), and if you put a space between the client and the opening parenthesis, those options apply not to the client but to everyone — a trap the man page explicitly warns about. After editing the file, tell the server with -ra of exportfs(8), and check with exportfs -v that it is actually being exported. NFSv4 works on 2049 alone, so as in this lab you can mount 127.0.0.1:/srv/share inside the same VM.

autofs attaches on access. The master map of auto.master(5) decides "which map for which directory", and the map file of autofs(5) decides "key, options, source". Ubuntu's /etc/auto.master has +dir:/etc/auto.master.d at the end, so the *.autofs files in that directory are read as master maps. After writing the map you have to restart the daemon, and ls /nfs/share is itself the trigger of the mount. One surprise: if the server is itself, autofs attaches with a bind mount instead of NFS. auto.master(5) describes the nobind option as "prevents bind mounting of local NFS filesystems", and that sentence tells you the default behavior.

What it looks like in the field

The question "I grew the disk, so why is the space the same?" comes up every week in the cloud too. Growing the volume (cloud console), growing the partition or LV, and growing the filesystem are the work of three layers, and someone who changed the size in the console has done only the first layer. If you log into the node and go from top to bottom — checking with lsblk whether the device grew, with lvs whether the LV grew, and with df whether the filesystem grew — you can see at once which layer it stopped at.

A Kubernetes ReadWriteMany volume is usually NFS. If a Pod stalls with mount.nfs: access denied by server, check whether the client range in the server's /etc/exports includes the node IP, and whether exportfs -v really has that entry. There are more cases than you would think where it is in the file but exportfs -ra was not run.

Disk performance problems are separated with iostat(1) and vmstat(8). For both, the first sample is a cumulative average since boot and must be discarded, and %util and await of iostat -x tell you whether the disk is busy, while the si/so columns of vmstat tell you whether it is hitting swap.

What you will do in the next lab

Two loop devices → PV → VG vg_lfcs → LV lv_data (ext4) → mount at /srv/data and a UUID fstab line → online extension → 256MiB swap file → export /srv/share over NFS to 127.0.0.1 only and mount it → the same share through autofs → iostat and vmstat metrics and a summary file. The grader does not look at files alone; it looks at the current system with vgs, lvs, findmnt, swapon --show, and exportfs -v. So do not stop at "I edited the file"; go on to "did the daemon learn of it".