Quarantining Incoming Media Before You Unpack It

Goal

Build a quarantine checker that reads only the listing of an incoming media's compressed file before unpacking to judge its scale, risky entries, compression ratio, and format mismatches, then filter each entry with an extraction filter, unpack into a quarantine directory, and write an import verdict with the basis hashes attached.

Why it matters

What comes into an air-gapped network is mostly media that a person carried in, and it contains a single compressed file. The one line that unpacks it to look inside is already an act of touching the file system. Absolute paths, parent-path escape, links pointing outside, special files, and decompression bombs all take effect the moment you unpack. A tar header records the name, size, permissions, entry type, and link target, so you can see all five of these without unpacking. So we change the order. Read first, judge, then unpack under conditions. A verdict must come with its basis. Only when it connects which hash of which archive was read as which report, and which reasons came out of that report, does the partner vendor know what to fix and can the reviewer issue the same verdict again. The grader does not trust your wording. It opens the archive itself and computes the values that should come out on its own, then reruns your quarantine checker according to the execution contract and compares even its output.

Steps

1. Create two pieces of media with the generation script and write the serial number, receipt time, submitter role, outer hash, and size in the intake register /root/media/intake.tsv.
2. Create /root/media/inspect.py, and without unpacking MEDIA-2609-017, read only the listing and write the entry count, total uncompressed size, and largest entry in /root/media/report/MEDIA-2609-017.json.
3. Add a danger classification to inspect.py. Provide five fields — absolute, traversal, escaping_link, special, and exec_bit — and leave fields with no matches as empty lists.
4. Add ratio and ratio_verdict to inspect.py. If the total uncompressed size divided by the compressed file size exceeds 100, it is a bomb.
5. Add mismatch to inspect.py. Write the entries whose format according to the extension differs from the format according to the leading bytes, as name, declared, and actual.
6. Add --extract-to to inspect.py, judge each entry with the data filter, unpack into /root/media/quarantine/MEDIA-2609-017, and write extracted and rejected in the report.
7. With /root/media/decide.py, turn the report into the import verdict /root/media/report/verdict-MEDIA-2609-017.json. Write media, archive_sha256, report_sha256, decision, reasons, accepted_entries, and rejected_entries.
8. Run the same quarantine checker and the same verdict rules on MEDIA-2609-018 to create /root/media/report/MEDIA-2609-018.json and /root/media/report/verdict-MEDIA-2609-018.json, and write /root/media/report/compare.tsv, which places the two pieces of media side by side.

Notes

• Execution contract: python3 /root/media/inspect.py --archive <매체> --out <보고서> [--extract-to <디렉터리>] (the placeholders are the media file, the report, and the directory). The verdict rules are python3 /root/media/decide.py --report <보고서> --out <판정서> (the placeholders are the report and the verdict). The grader reruns them itself according to this contract.
• The media are /root/media/in/MEDIA-2609-017.tar.gz and /root/media/in/MEDIA-2609-018.tar.gz, and the quarantine directory is /root/media/quarantine/<일련번호> (the placeholder is the serial number).
• Verdict rules (assumptions of this lab): if any of absolute, traversal, escaping_link, special, or bomb is present, reject; if the only reasons are exec_bit or mismatch, quarantine; if there is nothing, accept. reasons is a sorted list of reason names.
• Read the listing with tar -tvf, and in Python with tarfile.open(...).getmembers(). The per-entry verdict comes from tarfile.data_filter(member, dest).
• Look at the leading bytes with od -An -tx1 -N8 or Python. Some entries have NUL bytes mixed in, so you cannot see them properly with grep or head.
• Common mistakes: applying only a filter to extractall() and stopping at the first rejection, regenerating the report and leaving the verdict's hash as it is, and dropping the empty fields of danger altogether.
• This lab is air-gapped, so zip, unzip, xz, and gpg are not available. Use only tar and gzip.

Start with the intake register before opening the media

Create two pieces of media with the generation script and write one header line and two media lines in /root/media/intake.tsv. The fields are serial, received_at, submitter_role, sha256, and bytes, separated by tabs.

There is no sample to fetch from outside, so create it yourself with python3. The seed and mtime are fixed, so you get the same media no matter how many times you run it. Compute the hash on the file as it is, without unpacking the archive - this value is the only basis for later asking "is this the file we received back then?" Write the submitter as a role, not a person's name.

Read only the listing, without unpacking, to measure the scale

Have /root/media/inspect.py read only the headers of MEDIA-2609-017 without unpacking it and write archive_sha256, compressed_bytes, entries, total_uncompressed_bytes, largest_entry_bytes, and largest_entry_name to /root/media/report/MEDIA-2609-017.json.

tarfile.open(path, 'r:gz').getmembers() returns the list of headers as it is. Each entry's size is the value written in the header, so you do not need to read the contents. archive_sha256 is the hash of the archive file itself and compressed_bytes is the size of that file.

Split risky entries into five categories

Have inspect.py put danger in the report. Write lists of entry names in the five fields absolute, traversal, escaping_link, special, and exec_bit, and leave fields with no matching entries as empty lists.

If the name starts with a slash, it is an absolute path. Judge parent-path escape not by whether '..' appears in the string but by counting the depth component by component. Look at links by separating issym and islnk - a symbolic link is relative to where it sits, and a hard link is relative to the archive root. Special files are ischr, isblk, and isfifo.

Identify a bomb by compression ratio

Have inspect.py put ratio and ratio_verdict in the report. ratio is the total uncompressed size divided by the compressed file size, rounded to two decimal places; if it exceeds 100, ratio_verdict is bomb, otherwise ok.

Both values are already in the report. Do not measure by unpacking - that would mean unpacking the bomb in order to judge the bomb. If you pull the threshold out as a constant, the next person can change it to fit the organization's standard.

Check the extension against the actual format

Have inspect.py put mismatch in the report. Among regular files, write the entries whose format according to the extension differs from the format according to the leading bytes, as name, declared, and actual.

extractfile does not unpack to disk; it only opens a stream. The first 512 bytes are enough. gzip starts with 1f 8b and ELF starts with 7f 45 4c 46. If there is a NUL in the leading bytes, it is not text. Do not judge entries whose extension you do not know; skip them - if you write down what you do not know as wrong, the list becomes unusable.

Filter each entry with an extraction filter and unpack

Add --extract-to to inspect.py so that it judges each entry with the data filter and unpacks into /root/media/quarantine/MEDIA-2609-017. Write the extracted count and the rejected list (name, reason) in the report.

If you apply only filter='data' to extractall, an exception is raised at the first rejection and it stops. If you call tarfile.data_filter(member, dest) directly for each entry and catch FilterError, the rejected entries and their reasons are collected in one pass. Use the exception class name as the reason as it is. Collect only the entries that were not rejected and unpack them all at once at the end.

Write the import verdict with the basis hashes attached

Read the report with /root/media/decide.py and create /root/media/report/verdict-MEDIA-2609-017.json. Write media, archive_sha256, report_sha256, decision, reasons, accepted_entries, and rejected_entries.

The verdict rules are in '## Notes' of the instructions. reasons is a sorted list made of the names of the non-empty fields in danger, plus bomb and mismatch. report_sha256 is the value of sha256sum /root/media/report/MEDIA-2609-017.json - if you regenerate the report, you must rewrite this value too. This is not a step for fixing the blocking entries and letting them through.

Check whether the verdict splits on the second media

Run the same quarantine checker and the same verdict rules on MEDIA-2609-018 to create the report and verdict, and write /root/media/report/compare.tsv, which places the two pieces of media side by side. The header is media, entries, ratio, ratio_verdict, danger, mismatch, rejected, and decision.

Do not write anything new; use the two scripts you built earlier as they are. The danger field is the sum of the entry counts of all five categories. If a procedure does not react to values and always gives the same answer, it is not quarantine but a rubber stamp - check whether the decision on the two lines splits.