An Air Gap Does Not Take Your Tools Away — It Changes the Order

In one line

What makes an air-gapped network hard is not the absence of tools, but the fact that the investigation routine you always follow outside becomes entirely invalid.

Why this was needed

When you hit an unfamiliar error in an outside environment, this is the order of what you actually do: copy the error message. Search for it. Read what someone with the same symptom wrote. Paste the suggested command. If that fails, install one more tool. These five steps are so ingrained that we do not even call them "investigation." It is just how we work.

On a defense or public-sector defense site, all five steps disappear at once.

• There is no internet. No search, no documentation sites, no package repositories.
• You cannot bring a personal laptop in. The script you wrote yesterday is outside, and what is outside might as well not exist.
• You cannot take screenshots. Photos, captures, and personal storage media are all banned from being brought in.
• You cannot take logs out. Even if a log is essential to finding the cause, not a single line leaves before it passes review.

The most common mistake of someone entering for the first time is spending time lamenting what is missing. Only about 30 minutes later do they realize that method does not work in here, and they start over from that point. If three or four such 30-minute losses pile up in a day, the whole on-site schedule slips.

How it works

Investigation in an air-gapped network runs in the opposite order. Outside, you start from the symptom and go to knowledge. Inside, you start from what you have and go to the symptom.

First, before you begin, make a list of what you have. Which commands are installed, where the logs are and how many there are, whether configuration changes leave a record, where the configuration files are. This list is that day's investigation radius. If you start without a list, you keep trying methods outside the radius.

Second, pin down the fact that something is missing, with evidence. "The internet seems to be down" and "curl dies with exit code 28" are different sentences. The former makes someone try again later; the latter ends the discussion. A report that comes out of an air-gapped network has to describe what does not work more precisely than what does.

Third, narrow the cause by correlation. Since you cannot search for the meaning of an error code, about the only method left is to lay two different records from inside side by side on a time axis: the time of the first failure in the application log and the time a change was applied in the configuration change log. If the gap is 3 minutes, that is evidence. If the gap is three days, it is not. You make this judgment not with knowledge but with two sets of data and a clock.

Fourth, write things so they can be reproduced. An investigation script you build inside cannot leave, but it stays inside. To keep the next person from rebuilding the same thing, you have to leave it as a file on the spot. What is in your head walks out with you on the day the on-site assignment ends.

What it looks like in the field

Security pledges and media controls change the daily routine. When you enter, you hand over personal storage media. To bring code in, you have to put it on authorized media and pass an import review, and the review takes time. So in practice you often write things from scratch inside. That means a 20-line script you can improvise inside is more useful than a 100-line script you finished outside. Practice working with only standard tools, and with fewer of the tools you are used to, pays off here.

When the work ends, you return the media. The intermediate outputs you created during the work usually do not leave. So you have to separate "what goes out" from "what stays inside" from the very beginning. If you start choosing what to send out after everything is built, sensitive values are already mixed in everywhere.

One thing that burned me once. I once tried to take out the results of an air-gapped investigation and was rejected at export review because of a single internal hostname left in the summary. I had read the summary three times and still missed it. People cannot see what they themselves left behind in text they wrote themselves. Since then, I do not check by reading with my eyes; I check against a list of forbidden tokens extracted mechanically from the original.

What to read next

If what you have read so far was "how to find things inside," the piece that comes right after is "how to take out what you found." It covers why deciding what to keep and what to delete is itself a technique, in a situation where not a single line of the original log can leave but the cause still has to reach the headquarters development team. Then the next lab walks through the content of both pieces by hand in one go.