Building a Backup Set With tar

Goal

With tar, create a full backup and an incremental backup, and work through exclusion rules, permission preservation, and splitting.

Why it matters

A single tar option changes the restore result. If you include absolute paths without -C, you cannot choose the restore location; if you restore without --numeric-owner, owners are mapped incorrectly on another system; and if you restore without --selinux, the service does not start at all on an SELinux system. And because the snapshot file of --listed-incremental holds the state of the incremental chain, you must keep that file too — if you lose it, the next incremental effectively becomes a full backup.

Steps

Under /root/bk/src, create the following: app/main.py, app/conf.yaml, data/records.csv, logs/app.log, logs/old.log. Put at least one line in each file, but fill only data/records.csv to 200KB or more. In step 7 you split the archive into 64KB pieces, and if you put in only one-line files the compressed archive is under 1KB, so you get just one piece. Something like base64 /dev/urandom | head -c 240000 > data/records.csv is enough.
Create a full backup as /root/bk/full.tar.gz. Paths inside the archive must start with src/ (no absolute paths).
Write the number of archive entries as a single line of digits in /root/bk/count.txt.
Create incremental backups. The snapshot file is /root/bk/snap.snar. First create /root/bk/inc0.tar.gz, then add the file src/app/new.py and create /root/bk/inc1.tar.gz. The entry count of inc1 must be smaller than that of inc0.
Create a backup that excludes logs/, as /root/bk/nolog.tar.gz. logs must not appear in the listing.
Change the permission of /root/bk/src/app/conf.yaml to 640, create a full backup again (/root/bk/perm.tar.gz), and restore it to /root/bk/restore-perm/ with permissions preserved. Write the permission of the restored file as a single line of digits in /root/bk/perm.txt. It must be 640.
Split /root/bk/full.tar.gz into 64KB pieces, save them with the prefix /root/bk/split/full.part, then join them back to create /root/bk/rejoined.tar.gz. Its sha256 must match the original.
Create /root/bk/report.txt with the following 5 lines. FULL_ENTRIES=<3번 값> / INC0_ENTRIES=<inc0 항목 수> / INC1_ENTRIES=<inc1 항목 수> / NOLOG_HAS_LOGS=no / REJOIN_OK=yes (The placeholders stand for the value from step 3 and the entry counts of inc0 and inc1.)

Notes

If you build it in the form tar -czf /root/bk/full.tar.gz -C /root/bk src, the paths inside the archive become src/....
The entry count is tar -tzf <파일> | wc -l (the placeholder is the archive file).
An incremental is tar --listed-incremental=/root/bk/snap.snar -czf <출력> -C /root/bk src (the placeholder is the output file).
Splitting is split -b 64K <파일> /root/bk/split/full.part, and rejoining is cat /root/bk/split/full.part* > <출력> (the placeholders are the input file and the output file).
Common mistake 1: if you delete the snapshot file every time in step 4, every run remains a full backup.
Common mistake 2: if you leave out the permission-preserving option when restoring in step 6, the umask is applied and the value differs.

Create the backup source

Under /root/bk/src, create the following: app/main.py, app/conf.yaml, data/records.csv, logs/app.log, logs/old.log. Put at least one line in each file, but fill only data/records.csv to 200KB or more. In step 7 you split the archive into 64KB pieces, and if you put in only one-line files the compressed archive is under 1KB, so you get just one piece. Something like base64 /dev/urandom | head -c 240000 > data/records.csv is enough.

Mixing subdirectories and logs gives the exclusion rule in a later step something to act on.

Full backup

Create a full backup as /root/bk/full.tar.gz. Paths inside the archive must start with src/ (no absolute paths).

There is an option that keeps absolute paths out of the archive. That is what lets you choose the restore location.

Check the listing

Write the number of archive entries as a single line of digits in /root/bk/count.txt.

There is an option that shows the contents without extracting. Count the entries and write the count to the file.

Two incremental backups

Create incremental backups. The snapshot file is /root/bk/snap.snar. First create /root/bk/inc0.tar.gz, then add the file src/app/new.py and create /root/bk/inc1.tar.gz. The entry count of inc1 must be smaller than that of inc0.

The snapshot file holds the state. If you keep using the same snapshot file, the incremental chain continues.

Apply an exclusion rule

Create a backup that excludes logs/, as /root/bk/nolog.tar.gz. logs must not appear in the listing.

You can specify exclusion patterns multiple times or read them from a file. That entry must not appear in the resulting listing.

Check permission preservation

Change the permission of /root/bk/src/app/conf.yaml to 640, create a full backup again (/root/bk/perm.tar.gz), and restore it to /root/bk/restore-perm/ with permissions preserved. Write the permission of the restored file as a single line of digits in /root/bk/perm.txt. It must be 640.

There is an option that restores permissions and owners exactly. After restoring, compare with stat.

Split and rejoin the archive

Split /root/bk/full.tar.gz into 64KB pieces, save them with the prefix /root/bk/split/full.part, then join them back to create /root/bk/rejoined.tar.gz. Its sha256 must match the original.

split takes a prefix and creates the piece files. Rejoin them with cat.

Summarize the backup set

Create /root/bk/report.txt with the following 5 lines. FULL_ENTRIES=<3번 값> / INC0_ENTRIES=<inc0 항목 수> / INC1_ENTRIES=<inc1 항목 수> / NOLOG_HAS_LOGS=no / REJOIN_OK=yes

Collect the size and entry count of each archive. The values must be actual measurements.