Lakehouse Table Format — Understanding Apache Iceberg Through Its Metadata
Undo a region you deleted by mistake, and review the next day's data on a branch before publishing
Goal
After three days of commits, cause an accident that wipes out one region, read the state before the accident with time travel, name that point in time with a tag, and undo it with a rollback. Then write the next day's data first to a branch instead of main, review it, and publish by advancing main.
Why it matters
Every Iceberg commit adds a new snapshot without deleting old ones. So "yesterday's state" is not something you pull out of a backup but the file list of that snapshot, and undoing is not copying files but moving the pointer called main to an old snapshot. It takes a few seconds. But old snapshots are not forever. When snapshot expiration runs, anything older than the threshold is deleted, and a point in time for which you only wrote down the ID disappears. A tag gives a snapshot a name and a separate retention period to protect it from expiration. A branch is a tool in the opposite direction. You commit new data somewhere other than main first, and when the checks pass, you advance main to that snapshot. Readers never see the data before the checks (write-audit-publish).
Steps
- With /root/ice/tt/days.py (app
ice-tt-days), createlake.tt.orders(format-version 2) and load 2026-03-01, 02, and 03 with one commit per day. - With /root/ice/tt/accident.py (app
ice-tt-accident), runDELETE FROM lake.tt.orders WHERE region = 'busan'(an accident caused on purpose). - With /root/ice/tt/travel.py (app
ice-tt-travel), find the snapshot of the third commit (just before the accident), and write the row count and the Busan row count at that time, and the Busan row count now, to /root/ice/tt/out/travel.json. - With /root/ice/tt/tag.py (app
ice-tt-tag), attach the tagbefore_accidentto that snapshot withRETAIN 7 DAYS. - With /root/ice/tt/rollback.py (app
ice-tt-rollback), callrollback_to_snapshotto roll main back to that snapshot. - With /root/ice/tt/branch.py (app
ice-tt-branch), create the branchfixand write 2026-03-04 only to that branch. - With /root/ice/tt/publish.py (app
ice-tt-publish), callfast_forwardto advance main tofix. - In /root/ice/tt/report.md, write three sections,
## 타임트래블## 롤백과 태그## 브랜치(keep these headings as written; they stand for time travel, rollback and tags, and branches).
Notes
- You see snapshots with
SELECT * FROM lake.tt.orders.snapshots, the path main has travelled withlake.tt.orders.history, and the name labels withlake.tt.orders.refs. - Time travel:
SELECT … FROM 표 VERSION AS OF <스냅샷 ID 또는 태그·브랜치 이름>,TIMESTAMP AS OF '<시각>'. The placeholders stand for the table, the snapshot ID or the tag or branch name, and the timestamp. - Writing to a branch:
df.writeTo("lake.tt.orders.branch_fix").append()orINSERT INTO lake.tt.orders.branch_fix …in SQL. - Common mistakes: believing a rollback deletes the accident snapshot (it does not — expiration deletes it), and running snapshot expiration before attaching the tag.
- Official docs: Branching and Tagging · Spark Queries — Time travel · Spark Procedures — rollback_to_snapshot · fast_forward · Spark DDL — Branching and Tagging DDL
Three days, three commits
Create /root/ice/tt/days.py with the app name ice-tt-days, have it create lake.tt.orders (six columns, 'format-version' = '2'), and load 2026-03-01, 03-02, and 03-03 with one append() per day.
The three snapshots are linked by parent. The grader checks that the first three snapshots are appends that in turn added that day's row count. If you ran it several times and the snapshots multiplied, run DROP TABLE lake.tt.orders PURGE and start over.
The accident — a whole region disappears
Create /root/ice/tt/accident.py with the app name ice-tt-accident and run DELETE FROM lake.tt.orders WHERE region = 'busan'.
A DELETE is also a commit, so a fourth snapshot is created. The files that contained Busan rows are replaced by new files without Busan and enter the list, while the old files just drop out of the list but stay on disk. The grader checks that the fourth snapshot has the third as its parent and that there are no Busan rows.
Time travel — read just before the accident
Create /root/ice/tt/travel.py with the app name ice-tt-travel, have it find the snapshot ID of the third commit in lake.tt.orders.snapshots, count the total row count and the Busan row count at that time with VERSION AS OF and the Busan row count in the current table, and write them to /root/ice/tt/out/travel.json as {"snapshot_id", "rows_before", "busan_before", "busan_after"}.
VERSION AS OF only reads the manifest list of that snapshot. The files from before the accident are still on disk, so they are read as they are. A snapshot ID is a 19-digit integer, so do not copy it by hand; read it in the script and use it.
Tag — a name and a retention period for a point in time
Create /root/ice/tt/tag.py with the app name ice-tt-tag and attach a tag to the snapshot from step 3 with ALTER TABLE lake.tt.orders CREATE TAG before_accident AS OF VERSION <ID> RETAIN 7 DAYS (the placeholder stands for the snapshot ID).
A tag goes into the refs of the metadata with a name, a snapshot ID, and max-ref-age-ms. RETAIN 7 DAYS is 604,800,000 ms. The snapshot a tag points to is excluded from expiration. The grader looks at the three values in refs.
Rollback — only the pointer moves
Create /root/ice/tt/rollback.py with the app name ice-tt-rollback and call CALL lake.system.rollback_to_snapshot('lake.tt.orders', <ID>) with the snapshot the tag before_accident points to (the placeholder stands for the snapshot ID).
A rollback does not create a new snapshot; it only changes the snapshot main points to. The accident snapshot stays in the snapshots list, and "accident → just before the accident" is recorded in the path main has travelled (the snapshot-log). The grader looks at that record and at whether the accident snapshot remains.
Write to a branch first
Create /root/ice/tt/branch.py with the app name ice-tt-branch, run ALTER TABLE lake.tt.orders CREATE BRANCH fix, and then write the 2026-03-04 file only to fix with writeTo("lake.tt.orders.branch_fix").append().
A branch forks from main at the time of creation (the snapshot you rolled back to). When you commit to fix, only fix moves ahead and main stays as it is, so readers do not yet see the March 4 data. The grader checks that the head of fix has the rolled-back snapshot as its parent and added the March 4 row count.
Publish — advance main
Create /root/ice/tt/publish.py with the app name ice-tt-publish and call CALL lake.system.fast_forward('lake.tt.orders', 'main', 'fix').
fast_forward works only when main is an ancestor of fix (it is rejected if another commit slipped into main in the meantime). It moves only the main pointer to the head of fix, with no new snapshot. The grader checks that main and fix point to the same snapshot and that main has both Busan and March 4.
The undo procedure on one page
In /root/ice/tt/report.md, write three sections, ## 타임트래블 ## 롤백과 태그 ## 브랜치 (time travel, rollback and tags, and branches). In the first section, put the busan_before from step 3 as a number.
Write it as the sequence the on-call person would follow when the same accident happens at dawn. Also write what you should do first if it is a table on which expiration runs.