An audit does not ask what you did, but how you can show it

In one line

What actually trips you up in audit response is not work you did not do, but the state in which the work you did and the files that prove it are not connected. Binding in advance, for each line of a requirement, which part of which file is the evidence (an evidence index) is the core of the response, and that index must also carry the hash of the evidence file to avoid lying on its own later.

Why this was needed

The most common scene in the first audit after delivery goes like this. A line of a requirement is read out and the evidence is asked for. The person in charge answers, "we are doing that." Then the question comes back: where can we see it? From that moment they log into the server, open the configuration, and dig through logs. After 20 minutes the item is skipped, and it stays in the record as evidence not submitted. Not because there was no work done, but because nobody laid down in advance the path between the work done and the file.

Laying this path cannot be done the day before the audit. Evidence changes over time. Configuration files are overwritten at the next change, logs disappear by rotation, and inspection results are overwritten by the next inspection. So an index that writes down only paths is of no use; it must also pin down what that file was at that time. A hash goes in that place. If the file changes, the index reveals it by itself, and once revealed you can simply collect again. What is not revealed is the problem.

Public standards assume the same structure. NIST SP 800-171 Rev 3 groups protection requirements for material handled by partners into families, and NIST SP 800-53 Rev 5 sets a much broader list of controls and how to select them. Either way, what the document sets is "what must be satisfied," not "by which file you must show it." The table that connects the two has to be built by each site itself, and if it is not built, you end up patching over it with human memory every time. What needs to be managed when logs are used as evidence is covered separately by NIST SP 800-92.

How it works

First, make the evidence explain itself. If you keep which file is the evidence for which requirement in someone's head or in a separate spreadsheet, it soon drifts. If you attach a header to the evidence file itself so it states what it covers, building the index becomes a matter of sweeping a directory. Formats varying from file to file is unavoidable. In configuration files it attaches as comment lines, and in inspection result JSON as top-level keys. The reading side just has to handle both cases.

Second, put the hash in the index. A path and a collection time alone merely point to "the file as it is now." The moment you put in a hash, the index points to that file as it was then, and a verifier can count three things. Entries pointing to files that do not exist, entries whose hash is off, and requirements with no evidence at all. Three numbers summarize the state of the response.

Third, separate the strength of the evidence. Configuration files and policy documents show "it is set up that way." Logs and inspection results show "it actually worked that way." The two are not substitutes. That access restrictions were written in the configuration does not tell you that the configuration was actually applied and worked, and conversely, from logs alone you cannot tell whether it was an intended rule or a coincidence. If you decide for each requirement which kind is needed, the places where there is evidence but it is insufficient come to light. Those places are next quarter's work.

Fourth, rerun the collection. When you run the same script again, the index that comes out must be identical except for the collection time. If it is not, the index has something mixed in, such as order or time, that has nothing to do with the judgment, and then the two results cannot be compared. Material that cannot be compared cannot tell you what changed by next quarter either.

Fifth, review again when sending it out. Evidence files were gathered for internal use, so things that must not go out as they are are mixed in. Internal addresses, authentication means, personal contact details, and the like. Once you mask them the file content changes, so the hash changes too. An accident that often happens here is sending out a package that contains the masked copies while leaving the index with the original hashes. When the receiving side verifies it, everything comes out as hash mismatch, and the whole package falls under suspicion. The masked copy must carry the masked copy's hash.

What it looks like in the field

At one site, when I tried to send out again the index submitted last quarter as it was, I stopped. Verifying it as it was showed four entries broken. Two were files that disappeared during a cleanup, and two were files whose contents had changed. The changed ones were worse. The file is right there so it looks fine to a human eye, but the contents were not what we had relied on as evidence back then. Without the hash, we would have submitted it as is, and what the audit opened would have differed from what we explained.

Another thing I often see is places where there is evidence but it is insufficient. A typical case is an incident response procedure that is well written but has not a single record of actually drilling or responding. Looking at the index alone, one piece of evidence is attached and it is a green light, but what the requirement asks is not the existence of the procedure but the fact that the procedure is running. If you divide evidence by strength, such places come out as a list, and that list becomes next quarter's plan.

What you will do in the next lab

With 12 synthetic requirements and 14 evidence files, you build an evidence index, pin down the hashes, and use the index verifier to count three kinds of breaks. You divide the evidence into design and operational, judge what is lacking for each requirement, and rerun the same collection to confirm it is identical except for the collection time. Finally, you make masked copies using the export review rules and seal the index again with the masked copies' hashes, so that the verifier can say, looking at that package alone, that there are no breaks. The remaining gaps are not hidden but written into the table of contents as they are.