Looking Inside After You Unpack Is Already Too Late

In one line

Quarantining incoming media means reading only the header listing of an archive before unpacking to judge its scale and risk, and what you look at after unpacking is not inspection but incident investigation.

Why this was needed

Nothing comes into an air-gapped network over the network. So almost everything that comes in is carried in by a person. The moment a partner vendor's representative walks through the front gate with removable media and places it on the import review desk, it usually contains a single compressed file. Patches, configuration, dependency libraries, and installation instructions are bundled into one lump.

The most common thing a reviewer does here is unpack it first. You have to unpack it to see inside. But the act of unpacking an archive is already an act of touching the file system. If an entry's name is /etc/cron.d/agent-sync, it tries to write to the place that name points to, and if it is ../../etc/profile.d/agent.sh, it tries to write outside the unpacked directory. If an entry is a symbolic link, the link is placed first, and entries that come after it are written to the wrong place through the link. If special files are mixed in, device nodes get created. All of this happens inside a single line of command run "just to look inside."

And then there are decompression bombs. A file of repeated identical bytes can reach a compression ratio of several hundred times. The file on the media is 16KB but becomes 4MB when unpacked, and if you raise that ratio just a little, the review desk's temporary disk runs dry first. When the disk runs dry, the review itself stops, and a stopped review usually ends with "pass it through for now and deal with it later."

So we change the order. First read the listing, then judge, then unpack conditionally.

How it works

tar is a very simple format. One 512-byte header holds the name, size, permission bits, entry type, and link target, and the contents follow it. Thanks to this structure, there is a lot you can learn without writing the contents to disk.

• Scale. Adding up all the size values in the headers gives how many bytes it will be once unpacked. getmembers() in Python's tarfile returns these headers as they are. Dividing by the compressed file size gives the compression ratio, and if it exceeds a threshold, it is treated as a bomb. There is no correct answer for the threshold itself — this lab uses 100 times as an assumption.
• Risky entries. Does the name start with a slash (absolute path)? Counted by path component, does it go outside the base directory (parent-path escape)? Does a link target point outside? Is it a special file such as a device or a FIFO? Is the execute bit on? All five need only the header.
• Format mismatch. If the name is .csv but the first two bytes are 1f 8b, it is gzip, and if it is .txt but starts with 7f E L F, it is an executable. extractfile() only opens a stream, so you can read just the first few hundred bytes and close it. You must not peek with grep or head here — on a file with NUL bytes mixed in, GNU grep prints only a single "Binary file matches" line instead of the content.

Only when the judgment is finished do you unpack. From Python 3.12, you can apply a filter to extraction. PEP 706 lays out the background, and the gist is that the default behavior of extractall() was to trust the archive's metadata as is, and that was the site of a vulnerability reported in 2007 (CVE-2007-4559). There are three filters. fully_trusted is the old behavior, tar imitates GNU tar, and data cuts out Unix-specific features to extract most narrowly. The PEP states that the default stays fully_trusted with a warning in Python 3.12 and 3.13 and becomes data from 3.14.

There is one point worth noting here. The data filter does not reject absolute paths. As the PEP's description of the tar filter says, it strips the leading slash and then judges, so /etc/cron.d/agent-sync comes in without an error as etc/cron.d/agent-sync inside the quarantine directory. The result measured on the lab image was the same. So "we turned the filter on, so it is safe" and "this is what was on this media" are different sentences. What must be left in the import verdict is the latter.

매체(tar.gz) ──▶ getmembers()  ──▶ 규모·위험·압축비·형식 불일치  ──▶ 판정서
                     (풀지 않음)                │
                                                └─▶ data 필터로 항목별 판정 ──▶ 격리 디렉터리
                                                                              거절 목록(사유 포함)

Rejections also have to be collected in one pass. If you pass a filter to extractall(), an exception is raised at the first rejection and it stops, and you cannot see what else was there after that. If you call data_filter() directly for each entry and catch the exception, the rejected entries and their reasons are all left at once. That list is exactly what the reviewer needs when sending it back to the partner vendor.

What it looks like in the field

A verdict must come with its basis. A verdict that says only "rejected as risky" cannot be verified again, and the partner vendor does not know what to fix. It has to connect, in one line, which hash of which archive was read as which report, and which reasons came out of that report. If you regenerated the report, rewrite the verdict's hash too — the story of going through review twice for missing this one line also appears in the import bundle lab.

A signature cannot replace quarantine. The signature verification covered in the import bundle lab of the same course tells you "the sender is who they say they are." Even if the sender is right, the directory that person compressed may come with 4MB of build cache and a development symbolic link attached. You do both the signature and the structure check, in order.

The media itself is also something to manage. How to handle media once the import is finished is a matter of procedure, not technique. Media sanitization is covered by NIST SP 800-88 Rev.1, and the family of media protection controls by NIST SP 800-53 Rev.5. This lab does not imitate those procedures; it looks only as far as whether the quarantine record is in a shape that can be handed to them.

One thing that burned me once. There was a medium that someone unpacked at the review desk, judged "nothing special," and passed along. Later it turned out that docs/notes.txt inside was not text. There is no way for a person to see by eye whether a name matches its extension. Since then, the quarantine checker reads the leading bytes.

What you will do in the next lab

You build two pieces of media synthetically and first write the intake register, then without unpacking, read only the listing to measure the scale, split the risky entries into five categories, identify a bomb by compression ratio, and find mismatches between extension and actual format. Next, you judge each entry with the data filter, unpack into the quarantine directory, and leave a rejected list, then write an import verdict with the basis hashes attached. Finally, you run the same procedure unchanged on the second media and see whether the verdict flips.