Working With Versioning and Lifecycle

Goal

Cover in one go turning on versioning, how versions accumulate, how deletion is expressed as a delete marker, and the lifecycle rules that clean it up.

Why it matters

Versioning is a safety net for mistake recovery, but it is also a cost trap. If you turn on versioning for code that overwrites a log at the same key every day, after a year 365 versions have piled up on that one key. The list shows a single object, but the charge is 365 times. And the cause of the inquiry "I deleted everything but the charge is the same" is the delete marker — if you delete without a version ID, the data stays as it is and only one marker goes on top. That is why, in practice, the decision to turn on versioning and the decision to set lifecycle rules are always made together. Step 7 of this lab covers those three basic items.

Steps

Create the bucket lab-vers and turn on versioning. aws --profile local s3api get-bucket-versioning --bucket lab-vers must show Enabled.
Upload different content to s3://lab-vers/doc.txt 3 times. In the output of aws --profile local s3api list-object-versions --bucket lab-vers --prefix doc.txt, Versions must have 3 entries.
Specify the oldest version by version ID and download it to /root/vers/v1.txt. The content must be what was uploaded first.
Delete doc.txt without specifying a version. One DeleteMarkers entry must appear in the version list, making 4 entries in total. In /root/vers/marker.txt, write versions=4 has_delete_marker=true.
Delete the delete marker by specifying its version ID. aws --profile local s3 cp s3://lab-vers/doc.txt - must return the content of the 3rd upload. In /root/vers/restore.txt, write restored=true.
Write three lines in /root/vers/versions.txt: current_bytes=<n>, noncurrent_bytes=<n> and total_bytes=<n>. total is the sum of the other two values, and noncurrent must be greater than 0.
Put the lifecycle rules in /root/vers/lifecycle.json and apply them with aws s3api put-bucket-lifecycle-configuration --bucket lab-vers --lifecycle-configuration file:///root/vers/lifecycle.json. All three actions must be included — expire noncurrent versions after 30 days, clean up expired delete markers, and abort incomplete multipart uploads after 7 days. Save the output of aws s3api get-bucket-lifecycle-configuration --bucket lab-vers to /root/vers/ilm.txt.

Notes

Version list: aws s3api list-object-versions --bucket lab-vers --prefix doc.txt — in Versions the newest comes first, and IsLatest of each entry tells whether it is current. Delete markers come separately in DeleteMarkers.
Reading a specific version: aws s3api get-object --bucket lab-vers --key doc.txt --version-id <id> /root/vers/v1.txt
Deleting a specific version: aws s3api delete-object --bucket lab-vers --key doc.txt --version-id <id> (cannot be undone)
The shape of a version ID differs by server (AWS uses a random string of about 32 characters, and this server uses a hexadecimal string). Do not assume a format; pick it from the list.
This server (SeaweedFS) accepts lifecycle rules, stores them and returns them. You cannot see a 30-day expiry actually run within the time of this lab, so the grading looks only as far as whether the rules are set on the bucket.
Common mistake 1: deleting without a version ID and believing "it's deleted" — the data and the charges stay as they are.
Common mistake 2: turning on versioning only and not setting lifecycle — costs quietly accumulate.

Turn on bucket versioning

Create the bucket lab-vers and turn on versioning. aws --profile local s3api get-bucket-versioning --bucket lab-vers must show Enabled.

It is a per-bucket setting. After turning it on, query the status to confirm.

Write three times to the same key to accumulate versions

Upload different content to s3://lab-vers/doc.txt 3 times. In the output of aws --profile local s3api list-object-versions --bucket lab-vers --prefix doc.txt, Versions must have 3 entries.

Make the content different each time so that you can tell them apart later. Query the version list.

Read an old version

Specify the oldest version by version ID and download it to /root/vers/v1.txt. The content must be what was uploaded first.

Query by specifying the version identifier. The content uploaded first must come out.

Check the delete marker after deleting

Delete doc.txt without specifying a version. One DeleteMarkers entry must appear in the version list, making 4 entries in total. In /root/vers/marker.txt, write versions=4 has_delete_marker=true.

If you delete without specifying a version, nothing actually disappears. A new entry appears in the version list.

Delete the delete marker to recover

Delete the delete marker by specifying its version ID. aws --profile local s3 cp s3://lab-vers/doc.txt - must return the content of the 3rd upload. In /root/vers/restore.txt, write restored=true.

If you delete the marker, the version just below becomes current again. This is the mechanism of mistake recovery.

Calculate the capacity of noncurrent versions

Write three lines in /root/vers/versions.txt: current_bytes=<n>, noncurrent_bytes=<n> and total_bytes=<n>. total is the sum of the other two values, and noncurrent must be greater than 0.

The list shows a single object, but the charge applies to every version. Work out the total yourself.

Apply lifecycle rules

Put the lifecycle rules in /root/vers/lifecycle.json and apply them with aws s3api put-bucket-lifecycle-configuration --bucket lab-vers --lifecycle-configuration file:///root/vers/lifecycle.json. All three actions must be included — expire noncurrent versions after 30 days, clean up expired delete markers, and abort incomplete multipart uploads after 7 days. Save the output of aws s3api get-bucket-lifecycle-configuration --bucket lab-vers to /root/vers/ilm.txt.

Expiring noncurrent versions, cleaning up expired markers and aborting incomplete uploads are the basic set of three.