Storage in Practice — RAID, Snapshots, iSCSI, fio
Roll Back with a Snapshot, Grow XFS, Shrink ext4
Goal
Take an LVM snapshot right before a change, roll back a wrong change with a merge, grow XFS online, shrink ext4 offline while protecting the data, and create overprovisioning with a thin pool.
Why it matters
Change work is approved on the premise that "if it goes wrong, we roll back." A snapshot creates that premise in a few seconds, but if you size it wrong it quietly becomes invalid during the work, and if you do not know the rollback procedure (unmount → merge → mount again), you flounder at dawn. The choice of filesystem is the same — XFS is designed as something that can be grown but not shrunk, so what you chose for a volume that might need shrinking decides your options later.
This lab runs as root on an Ubuntu 24.04 VM. Since there are no empty disks, you attach a 3GiB sparse file as a loop device and use it in place of a disk. The session starts at 60 minutes and can be extended up to 180 minutes, and the VM disappears when the session ends.
Steps
- Create
/var/lib/sto/lvm.imgas 3GiB, attach it as a loop device, initialize it as a PV, and create the volume groupvg_sto. - Format a 600MiB LV
lv_appas XFS and mount it at/srv/app. Writeversion=1to/srv/app/app.confand a CSV of three or more lines to/srv/app/orders.csv, and save the output ofsha256sum /srv/app/orders.csvto/root/sto/snap/orders.sha256. - Create a snapshot
lv_app_snapof 200MiB, and save the output oflvs -o lv_name,origin,lv_attr,lv_size vg_stoto/root/sto/snap/snap.txt. - The wrong change: set
app.conftoversion=2, deleteorders.csv, and write 60MiB to/srv/app/junk.bin. Then save the output oflvs -o lv_name,origin,data_percent vg_stoto/root/sto/snap/usage.txt. - Merge the snapshot to roll back.
lv_app_snapmust be gone,/srv/appmust be mounted again, and you must haveversion=1, a matchingorders.csvchecksum, and nojunk.bin. - Grow
lv_appto 900MiB and grow the XFS too, while it stays mounted. - Mount a 500MiB LV
lv_logsas ext4 at/srv/logs, write 5MiB to/srv/logs/keep.log, and then save the output ofsha256sum /srv/logs/keep.logto/root/sto/snap/keep.sha256. - Shrink
lv_logsto 300MiB. The filesystem must shrink with it, it must be mounted again, andkeep.logmust match the checksum. - On a 400MiB thin pool
pool, create a thin volumelv_thinwith a virtual size of 1GiB, and save the output oflvs -o lv_name,lv_size,pool_lv,data_percent vg_stoto/root/sto/snap/thin.txt.
Notes
- Snapshot:
lvcreate -s -L 200M -n lv_app_snap vg_sto/lv_app; rollback:lvconvert --merge. - Growing:
lvextend -r -L 900M vg_sto/lv_app. Shrinking:e2fsck -f→resize2fs <작게>→lvreduce→resize2fs(the placeholder stands for a size smaller than the target). - Thin:
lvcreate --type thin-pool,lvcreate -V 1G -T vg_sto/pool -n lv_thin. - Common mistake 1: running
lvconvert --mergewhile mounted and wondering "why didn't it go back" — the merge was deferred until the next activation. - Common mistake 2: doing
lvreducefirst andresize2fsafterward — the end of the filesystem gets cut off.
Build a VG on a loop device
Create /var/lib/sto/lvm.img as a 3GiB sparse file, attach it as a loop device, initialize it as a PV, and create the volume group vg_sto from that one PV.
The order is truncate → losetup -f --show → pvcreate → vgcreate. You find the loop number again with losetup -j .
An XFS volume and the data to protect
Create a 600MiB LV lv_app in vg_sto, format it as XFS, and mount it at /srv/app. Write the single line version=1 to /srv/app/app.conf and any CSV of three or more lines to /srv/app/orders.csv, and save the output of sha256sum /srv/app/orders.csv to /root/sto/snap/orders.sha256.
The order is lvcreate -L 600M -n lv_app vg_sto, mkfs.xfs, mount. The checksum file must contain the absolute path so that you can check it with sha256sum -c later.
Snapshot right before the change
Create a snapshot lv_app_snap of lv_app with a size of 200MiB. Then save the output of lvs -o lv_name,origin,lv_attr,lv_size vg_sto to /root/sto/snap/snap.txt.
It is lvcreate -s -L -n vg_sto/lv_app. The lv_attr of a snapshot starts with s, and the origin name shows in the Origin column.
The wrong change and snapshot usage
Imitate a wrong change: change /srv/app/app.conf to version=2, delete /srv/app/orders.csv, and write 60MiB to /srv/app/junk.bin. Then save the output of lvs -o lv_name,origin,data_percent vg_sto to /root/sto/snap/usage.txt.
You can write it with dd if=/dev/urandom of=... bs=1M count=60. You have to run sync after writing for it to be reflected in the snapshot usage (Data%). If you write more than the snapshot can hold, the snapshot becomes invalid, so stay within the size.
Roll back with a merge
Merge the snapshot into the origin to roll back to before the wrong change. When finished, lv_app_snap must be gone, /srv/app must be mounted again, app.conf must be version=1, orders.csv must match the checksum, and junk.bin must not exist.
If the origin is mounted, lvconvert --merge is deferred until the next activation. Unmount first, merge, and then mount again.
Grow XFS while it stays mounted
Grow lv_app to 900MiB, and grow the XFS as well without unmounting. /srv/app as seen by df must get bigger.
lvextend -r grows the LV and then calls the tool that fits the filesystem (xfs_growfs for XFS). xfs_growfs works only on a mounted filesystem.
Create a volume that can be shrunk
Create a 500MiB LV lv_logs in vg_sto, format it as ext4, and mount it at /srv/logs. Write 5MiB of random data to /srv/logs/keep.log and save the output of sha256sum /srv/logs/keep.log to /root/sto/snap/keep.sha256.
It is the same flow as step 2, with only the filesystem being ext4. You shrink this volume in the next step.
Shrink ext4 while protecting the data
Shrink lv_logs to 300MiB. The filesystem must shrink with it, and when finished it must be mounted again at /srv/logs and keep.log must match the checksum.
ext4 can shrink only while unmounted. The order matters — first shrink the filesystem to smaller than the target (e2fsck -f is needed first), then shrink the LV, and finally resize the filesystem again to fit the LV size.
Thin pool and overprovisioning
Create a 400MiB thin pool pool in vg_sto, and on that pool create a thin volume lv_thin with a virtual size of 1GiB. Save the output of lvs -o lv_name,lv_size,pool_lv,data_percent vg_sto to /root/sto/snap/thin.txt.
Create the pool with lvcreate --type thin-pool -L, and the thin volume with lvcreate -V -T vg_sto/pool -n . If you create a volume larger than the pool, an overprovisioning warning appears, and that is what this step is meant to show.