Generational Backups With rsync

Goal

With rsync, set up mirror synchronization and generation backups, and build them into a production-ready form, including a --delete guard and load control.

Why it matters

The --link-dest generation backup is rsync's most elegant feature. Each generation directory looks like a complete snapshot, while the disk is occupied only by the changes. The principle is hard links, which is why measuring with du gives a much smaller number than you would expect — it does not count the same inode twice.

And --delete must always be used with a guard. If it runs while the source is empty or the mount has come undone, it deletes all the data on the target. It is an accident that genuinely keeps recurring in practice.

Steps

1. Under /root/rs/src, create app/main.py, conf/app.yaml, logs/app.log, and tmp/cache.bin, and synchronize to /root/rs/dst/. As a result, /root/rs/dst/app/main.py must exist.
2. Run it as dry-run plus itemize and save the output to /root/rs/itemize.txt. The target must be a new directory that has not been synchronized yet — if you aim at the dst/ from step 1 again, it is already in the same state, so rsync prints no lines and the file ends up empty. Itemize is a tool for seeing "what changes and why," so there has to be something that will change for there to be something to see.
3. Write /root/rs/backup.sh. It must start with set -euo pipefail, abort with exit code 1 if the source is empty, and use --delete to synchronize /root/rs/src/ to /root/rs/mirror/. The source path is taken as the first argument, and if there is no argument it uses /root/rs/src/ — the grader passes an empty directory as the first argument to confirm that the guard actually works. Run it to create /root/rs/mirror.
4. Write the exclusion patterns in /root/rs/exclude.txt (logs/ and tmp/), and use it to synchronize to /root/rs/clean/. /root/rs/clean/logs and /root/rs/clean/tmp must not exist.
5. Create a generation backup. After a full synchronization to /root/rs/gen/daily.1/, use --link-dest to create /root/rs/gen/daily.0/. Files that did not change must share the same inode between the two generations.
6. Use --numeric-ids to synchronize to /root/rs/numeric/, and write the owner UID of app/main.py as a single line of digits in /root/rs/uid.txt.
7. Write /root/rs/throttled.sh. It must be an rsync command that includes all three of ionice, nice, and --bwlimit. Run it to create /root/rs/slow/.
8. Create /root/rs/report.txt with the following 4 lines. MIRROR_FILES=<mirror 안 파일 수> / CLEAN_HAS_LOGS=no / SHARED_INODE=yes / GEN_COUNT=2 (The placeholder stands for the number of files inside mirror.)

Notes

• rsync -ain <원본>/ <대상>/ is the dry-run plus itemize combination (the placeholders are the source and the destination).
• Specify the exclusion list with --exclude-from=/root/rs/exclude.txt.
• To check inode sharing, compare with stat -c %i <두 세대의 같은 파일> (the placeholder is the same file in the two generations).
• Load control example: ionice -c 3 nice -n 19 rsync -a --bwlimit=1M <원본>/ <대상>/ (the placeholders are the source and the destination).
• Common mistake 1: the script in step 3 hardcodes the source path and so fails the empty-directory test. Write it to take the source path as the first argument (use a default when there is no argument).
• Common mistake 2: in step 5, if you give --link-dest as a relative path it is interpreted relative to the destination, and the links are not created.

Basic mirror synchronization

Under /root/rs/src, create app/main.py, conf/app.yaml, logs/app.log, and tmp/cache.bin, and synchronize to /root/rs/dst/. As a result, /root/rs/dst/app/main.py must exist.

Remember the trailing slash rule. The source directory name must not appear inside the destination.

Show the changes

Run it as dry-run plus itemize and save the output to /root/rs/itemize.txt. The target must be a new directory that has not been synchronized yet — if you aim at the dst/ from step 1 again, it is already in the same state, so rsync prints no lines and the file ends up empty. Itemize is a tool for seeing "what changes and why," so there has to be something that will change for there to be something to see.

Using dry-run and itemize together prints, one line at a time, what will change and how.

A backup script with a guard

Write /root/rs/backup.sh. It must start with set -euo pipefail, abort with exit code 1 if the source is empty, and use --delete to synchronize /root/rs/src/ to /root/rs/mirror/. The source path is taken as the first argument, and if there is no argument it uses /root/rs/src/ — the grader passes an empty directory as the first argument to confirm that the guard actually works. Run it to create /root/rs/mirror.

If the source is empty, it must stop immediately. Distinguish the cases by exit code.

Exclusion list file

Write the exclusion patterns in /root/rs/exclude.txt (logs/ and tmp/), and use it to synchronize to /root/rs/clean/. /root/rs/clean/logs and /root/rs/clean/tmp must not exist.

Collecting the patterns in a file makes them easy to manage. One pattern per line.

Generation backup

Create a generation backup. After a full synchronization to /root/rs/gen/daily.1/, use --link-dest to create /root/rs/gen/daily.0/. Files that did not change must share the same inode between the two generations.

It is safer to give --link-dest as an absolute path. Verify by the link count.

Keep numeric owners

Use --numeric-ids to synchronize to /root/rs/numeric/, and write the owner UID of app/main.py as a single line of digits in /root/rs/uid.txt.

There is an option that uses numeric IDs instead of names. It is a safeguard for when the restore target system is different.

Load control

Write /root/rs/throttled.sh. It must be an rsync command that includes all three of ionice, nice, and --bwlimit. Run it to create /root/rs/slow/.

Use the bandwidth limit option together with the command that lowers disk priority. Save the entire command as a script.

Per-generation size report

Create /root/rs/report.txt with the following 4 lines. MIRROR_FILES=<mirror 안 파일 수> / CLEAN_HAS_LOGS=no / SHARED_INODE=yes / GEN_COUNT=2

Because of hard links, the du result is smaller than you would expect. That fact itself is the conclusion of this lab.