TT Lab
Get started
Learn Learning paths Courses

Storage in Practice — RAID, Snapshots, iSCSI, fio

Roll Back with a Snapshot, Grow XFS, Shrink ext4

Continue in TT Lab

Goal

Take an LVM snapshot right before a change, roll back a wrong change with a merge, grow XFS online, shrink ext4 offline while protecting the data, and create overprovisioning with a thin pool.

Why it matters

Change work is approved on the premise that "if it goes wrong, we roll back." A snapshot creates that premise in a few seconds, but if you size it wrong it quietly becomes invalid during the work, and if you do not know the rollback procedure (unmount → merge → mount again), you flounder at dawn. The choice of filesystem is the same — XFS is designed as something that can be grown but not shrunk, so what you chose for a volume that might need shrinking decides your options later.

This lab runs as root on an Ubuntu 24.04 VM. Since there are no empty disks, you attach a 3GiB sparse file as a loop device and use it in place of a disk. The session starts at 60 minutes and can be extended up to 180 minutes, and the VM disappears when the session ends.

Steps

  1. Create /var/lib/sto/lvm.img as 3GiB, attach it as a loop device, initialize it as a PV, and create the volume group vg_sto.
  2. Format a 600MiB LV lv_app as XFS and mount it at /srv/app. Write version=1 to /srv/app/app.conf and a CSV of three or more lines to /srv/app/orders.csv, and save the output of sha256sum /srv/app/orders.csv to /root/sto/snap/orders.sha256.
  3. Create a snapshot lv_app_snap of 200MiB, and save the output of lvs -o lv_name,origin,lv_attr,lv_size vg_sto to /root/sto/snap/snap.txt.
  4. The wrong change: set app.conf to version=2, delete orders.csv, and write 60MiB to /srv/app/junk.bin. Then save the output of lvs -o lv_name,origin,data_percent vg_sto to /root/sto/snap/usage.txt.
  5. Merge the snapshot to roll back. lv_app_snap must be gone, /srv/app must be mounted again, and you must have version=1, a matching orders.csv checksum, and no junk.bin.
  6. Grow lv_app to 900MiB and grow the XFS too, while it stays mounted.
  7. Mount a 500MiB LV lv_logs as ext4 at /srv/logs, write 5MiB to /srv/logs/keep.log, and then save the output of sha256sum /srv/logs/keep.log to /root/sto/snap/keep.sha256.
  8. Shrink lv_logs to 300MiB. The filesystem must shrink with it, it must be mounted again, and keep.log must match the checksum.
  9. On a 400MiB thin pool pool, create a thin volume lv_thin with a virtual size of 1GiB, and save the output of lvs -o lv_name,lv_size,pool_lv,data_percent vg_sto to /root/sto/snap/thin.txt.

Notes

Build a VG on a loop device

Create /var/lib/sto/lvm.img as a 3GiB sparse file, attach it as a loop device, initialize it as a PV, and create the volume group vg_sto from that one PV.

The order is truncate → losetup -f --show → pvcreate → vgcreate. You find the loop number again with losetup -j .

An XFS volume and the data to protect

Create a 600MiB LV lv_app in vg_sto, format it as XFS, and mount it at /srv/app. Write the single line version=1 to /srv/app/app.conf and any CSV of three or more lines to /srv/app/orders.csv, and save the output of sha256sum /srv/app/orders.csv to /root/sto/snap/orders.sha256.

The order is lvcreate -L 600M -n lv_app vg_sto, mkfs.xfs, mount. The checksum file must contain the absolute path so that you can check it with sha256sum -c later.

Snapshot right before the change

Create a snapshot lv_app_snap of lv_app with a size of 200MiB. Then save the output of lvs -o lv_name,origin,lv_attr,lv_size vg_sto to /root/sto/snap/snap.txt.

It is lvcreate -s -L -n vg_sto/lv_app. The lv_attr of a snapshot starts with s, and the origin name shows in the Origin column.

The wrong change and snapshot usage

Imitate a wrong change: change /srv/app/app.conf to version=2, delete /srv/app/orders.csv, and write 60MiB to /srv/app/junk.bin. Then save the output of lvs -o lv_name,origin,data_percent vg_sto to /root/sto/snap/usage.txt.

You can write it with dd if=/dev/urandom of=... bs=1M count=60. You have to run sync after writing for it to be reflected in the snapshot usage (Data%). If you write more than the snapshot can hold, the snapshot becomes invalid, so stay within the size.

Roll back with a merge

Merge the snapshot into the origin to roll back to before the wrong change. When finished, lv_app_snap must be gone, /srv/app must be mounted again, app.conf must be version=1, orders.csv must match the checksum, and junk.bin must not exist.

If the origin is mounted, lvconvert --merge is deferred until the next activation. Unmount first, merge, and then mount again.

Grow XFS while it stays mounted

Grow lv_app to 900MiB, and grow the XFS as well without unmounting. /srv/app as seen by df must get bigger.

lvextend -r grows the LV and then calls the tool that fits the filesystem (xfs_growfs for XFS). xfs_growfs works only on a mounted filesystem.

Create a volume that can be shrunk

Create a 500MiB LV lv_logs in vg_sto, format it as ext4, and mount it at /srv/logs. Write 5MiB of random data to /srv/logs/keep.log and save the output of sha256sum /srv/logs/keep.log to /root/sto/snap/keep.sha256.

It is the same flow as step 2, with only the filesystem being ext4. You shrink this volume in the next step.

Shrink ext4 while protecting the data

Shrink lv_logs to 300MiB. The filesystem must shrink with it, and when finished it must be mounted again at /srv/logs and keep.log must match the checksum.

ext4 can shrink only while unmounted. The order matters — first shrink the filesystem to smaller than the target (e2fsck -f is needed first), then shrink the LV, and finally resize the filesystem again to fit the LV size.

Thin pool and overprovisioning

Create a 400MiB thin pool pool in vg_sto, and on that pool create a thin volume lv_thin with a virtual size of 1GiB. Save the output of lvs -o lv_name,lv_size,pool_lv,data_percent vg_sto to /root/sto/snap/thin.txt.

Create the pool with lvcreate --type thin-pool -L, and the thin volume with lvcreate -V -T vg_sto/pool -n . If you create a volume larger than the pool, an overprovisioning warning appears, and that is what this step is meant to show.