Saying nothing goes out is not evidence

In one line

When you put a product into a network-separated environment, the sentence "nothing goes out" is not accepted. Only material that presents together what the configuration files are set up to call, what actually remained in the records, and how far the scope you could not confirm extends counts as evidence.

Why this was needed

Near the end of delivery, the question the audit team asks is always similar. "What does this product send outside?" The person who built it wants to answer "it sends nothing." But today's products almost all include update checks, online license checks, remote diagnostic uploads, crash reports, usage statistics, and time synchronization. Most are on by default, and most are not in the table of the installation document. In a development environment there is an internet connection, so years pass with nobody noticing.

The problem is that this question is not a matter of trust but a matter of procedure. The audit team is not asking because it doubts the person who built it. A promise made in words is not passed on to the next person in charge, and it is quietly reversed at the next version upgrade. So it demands a form that can be left as material. SP 800-53 Rev.5 from the US National Institute of Standards and Technology puts boundary protection in the System and Communications Protection (SC) family and states "let through only what is permitted and block the rest" as the default stance. SP 800-171 Rev.3, which covers the defense supply chain, and SP 800-190, which covers container environments, also assume "deny communication by default" at the same place. Under this premise, the material we can submit is the full list of "what our product calls."

How it works

You read the three layers separately and link them by time.

First, configuration. What is the product set up to call? Two things deceive people every time here. One is items blocked off with comments. They are not deleted but only hidden, so they come back to life at the next version upgrade. So you list them in the table as off, but you do not remove them from the list. The other is items that are on by default. A common rule is that if enabled is absent from the configuration file altogether, the product is treated as on, and if you do not know this, the item drops out of the list entirely.

This second trap happens the same way on the tool side. The jq alternative operator // uses the right-hand side not only when the left is null but also when it is false. So .enabled // true looks like it means "on if the key is absent," but in fact it turns even an item that was clearly turned off with "enabled": false into an on item. A missing key and a value of false are different facts, and telling the two apart is half of this job.

Second, records. Proxy access records count actual attempts. You must not lump the response codes together here. 200 means the request really went out, 407 means the proxy demanded authentication and it stopped, and 403 means the proxy blocked it. Ten lines of 403 is not "nothing was sent" but "it tried to go out ten times and was blocked ten times." The fact to submit to the audit is the latter.

Third, name resolution. Communication that does not go through the proxy leaves nothing in the proxy records. Time synchronization over UDP, libraries that do not read the proxy settings, and direct connections dropped straight at the firewall are all like this. Such features still ask for names. So destinations that appear only in the query records and not in the access records turn up separately, and they are the only trace that that code path actually ran. Conversely, if an address is written directly in the configuration, it does not remain in the query records either. If it is a private-range address from RFC 1918, the chance that it went outside is low, but as long as the allowlist is written by name, you cannot check which service that address is. Write such an item as cannot judge, neither allowed nor disallowed, and leave it under what you could not confirm.

And time. If you link the three records in one table, you get "since when did this attempt exist?" If the time of the configuration change log and the time of the first query are close together, that is a basis for saying that change is the cause. This table is useful in the opposite direction too. A destination that appears only in the records and not in the change log was left by an installation script or a person's hand, and reveals what happened outside configuration management.

Assumptions of this lab

The lab Pod has no kernel permissions at all. Neither tcpdump nor iptables works, so there is no way to capture and show packets in the first place. In real delivery sites too, developers almost never get access to the customer's firewall, and that place is filled by the firewall policy copy and device logs that the network team hands over. This lab substitutes the proxy access records and the name resolution records for that place. The product rule in the configuration file that "a section without enabled is treated as on" is also an assumption of this lab, and since it differs from product to product, you must confirm it in the manual in the field.

What it looks like in the field

In one delivery, "we will turn telemetry off" was written in the meeting minutes and it was actually turned off, but half a year later a version upgrade replaced the default configuration file wholesale and it came back to life. The reason nobody knew is simple. The fact that it had been turned off existed only in people's memory and the minutes, and not in a place a machine checks. If you leave the list and the re-collected records as material, you can run the same procedure once more at the next version upgrade.

In another case, I saw someone delete the relevant lines from the records to produce "0 disallowed." That is not material but the opposite of material. The way to show absence is not to delete but to place two sets of records, before turning off and after turning off, side by side. The approved destination still remaining in the after-record also shows that the collection was genuine.

What really matters in practice

• Write the scope you confirmed and what you could not confirm in the same document. An "all clear" that does not state its scope deceives the next person.
• Do not write only the verdict; leave the counted values and which file they came from. The audit officer must be able to recount on their own.
• For a feature that is truly needed, redirect it to an approved internal destination instead of turning it off. If you turn off time synchronization, an accident happens elsewhere.
• Material goes stale. Turn the procedure into a script so it can be run again at every version upgrade, and hand it over.

What you will do in the next lab

You extract every address pointing outside from three configuration files, separate on from off, and check against the allowlist to split them into three branches: allowed, disallowed, and cannot judge. You count attempts in the proxy records by response, and separately find the destinations that appear only in the query records. After linking the three records by time and pinpointing which configuration change produced those attempts, you actually turn the configuration off, re-collect the records, and show that they disappeared. At the end you submit the audit summary both as machine-readable JSON and as a human-readable table.