The batch ran before the queue — the clock was 2.4 seconds fast

Goal

Gather the logs of five hosts with differing notations onto one axis, find the host isolated from the time hierarchy, find the lower bound of the clock skew from the contradictions between requests and responses, and then separate what is explained by correction from what is not and write a time reliability statement.

Why it matters

Both investigations and audits ask "what happened first?" The answer comes from the times of the records, and the time of a record is the value shown by the clock of the machine that wrote it. A clock off by about 2 seconds flips cause and effect without anyone noticing. In an air-gapped network there is no outside time source and everyone aligns to one time source inside the network, so their mutual order matches but absolute time is not guaranteed. If you do not write that limit in the statement, the whole document falls apart later.

Why the judgment is made from data

The lab Pod has no capability, so you cannot use chronyc or ntpdate and cannot change the system clock. So this lab judges from the synchronization status report and logs alone. In real sites too, what you submit to the audit is not command output but the result of this judgment, and the output of chronyc tracking and sources occupies the place where sync.json sits here. This is an assumption of this lab.

Steps

1. With python3, create sync.json, meta.json, five logs, and requests.csv in /root/clock/data.
2. Gather the five logs in /root/clock/normalized.csv as host,seq,epoch_ms,evt, and write the impact of wrongly reading the logs without offsets as UTC in /root/clock/naive.json.
3. Write each host's time source chain in /root/clock/chain.json, and the hosts that do not reach the declared time source in /root/clock/isolated.txt.
4. Write the cases where the response comes before the request in /root/clock/paradox.csv, and the lower bound of the clock skew in /root/clock/bound.json.
5. Write the places where the time goes backward within one host's log in /root/clock/backward.csv.
6. Write the times corrected by offsets in /root/clock/corrected.csv, and the number of contradictions before and after correction in /root/clock/paradox_after.json.
7. Write the record trust level for each host in /root/clock/trust.csv as host,level,reason.
8. Write the time reliability statement in /root/clock/statement.json and /root/clock/statement.md.

Notes

• The log notation rules are in /root/clock/data/meta.json. If you read a log without an offset as UTC, it shifts by nine hours.
• Apply the step 7 verdict rules in order. If the chain is not reached, 설명안됨 (Korean for "unexplained"); otherwise, if the log has a backward jump, 설명안됨; otherwise, if a contradiction involving that host remains even after correction, 설명안됨; otherwise, if it was ever involved in a contradiction before correction, 보정으로설명됨 (Korean for "explained by correction"); the rest are 확인됨 (Korean for "confirmed"). The reason values are, in order, 계통미도달·역행·보정후모순·보정필요·없음 (the Korean words for "chain not reached," "went backward," "contradiction after correction," "correction needed," and "none").
• The correction is 참값 = 관측값 - offset_ms (true value equals observed value minus offset_ms). If you flip the sign, the contradictions actually increase.
• Common mistake 1: making epoch_ms a real number. Round and write it as an integer number of milliseconds.
• Common mistake 2: writing synchronization as the same thing as timestamp proof. They are different layers, and RFC 3161 covers the latter.
• Standards documents: RFC 5905, RFC 3339, NIST SP 800-92.

Create the synchronization report and the logs of five hosts

With python3, create sync.json, meta.json, node-a.log through node-e.log, and requests.csv in /root/clock/data. Use the generation script that uses no random numbers, as it is.

There is no sample to download in an air-gapped network, so create the data yourself first. Only if it uses no random numbers does everyone get the same data no matter who runs it how many times, and you can check one another's judgments against each other. The grader converts the data to a canonical form and checks fingerprints, so if you edit it by hand, all the later steps get blocked.

Gather the four notations onto one axis

In /root/clock/normalized.csv, put host,seq,epoch_ms,evt on the first line and write all the events of the five logs. In /root/clock/naive.json, write naive_utc_hosts, shifted_events, and shift_hours.

epoch_ms is an integer number of milliseconds in UTC. Those with an offset can be read as they are, and those without an offset have to be read in the time zone that meta.json tells you. How many hours it shifts if you read them as UTC is the content of naive.json.

Follow the time source hierarchy to the end

In /root/clock/chain.json, write as a list the time source chain followed for each host, and in /root/clock/isolated.txt, write the hosts that do not reach the declared time source, one per line.

The chain starts from that host's source and is followed step by step. If you meet LOCAL or meet a name you have already passed, stop there and include that name in the chain. Exclude the declared time source itself from the isolation judgment.

Find the lower bound of the skew from cases where the response comes before the request

In /root/clock/paradox.csv, put req_id,sender,receiver,delta_ms on the first line and write the cases where delta_ms is negative, and in /root/clock/bound.json, write min_skew_ms, sender, and receiver.

delta_ms is the receive time minus the send time. If it is negative, it means the two clocks are apart by at least that magnitude, and the magnitude of the largest negative is the lower bound. Write the lower bound as a positive number.

Find places where the time goes backward within one host

In /root/clock/backward.csv, put host,seq,prev_epoch_ms,epoch_ms on the first line, and write the points in the same host's log where the time becomes smaller than the immediately preceding line.

Read host by host in seq order and compare with the time of the immediately preceding line. Look at them separately so the hosts do not get mixed. Records in a stretch where the time went backward cannot be used for order as evidence.

Correct by offsets and see what remains

In /root/clock/corrected.csv, put host,seq,epoch_ms on the first line and write the corrected times, and in /root/clock/paradox_after.json, write before, after, and remaining.

Correction means subtracting that host's offset_ms from the observed value. If you flip the sign, the contradictions actually increase. If you correct both sides of the requests the same way and count the negatives again, what remains is the real defect that the clock does not explain.

Grade the record trust level for each host

In /root/clock/trust.csv, put host,level,reason on the first line and judge the five hosts that have logs. Apply the rules in the order given in the lab instructions.

Only the first rule that matches is applied. A host whose chain is not reached is unexplained without looking at any other basis. If it was involved in a contradiction before correction but it disappeared after correction, that is a stretch that can be used conditionally.

Write a statement that states both what can be trusted and what cannot

In /root/clock/statement.json, write root, root_has_external_reference, confirmed, corrected, unexplained, min_skew_ms, and remaining_contradictions, and in /root/clock/statement.md, write at least 500 characters in four sections, ## 판단, ## 근거, ## 믿을 수 없는 것, and ## 남은 의심 (the Korean headings mean, in order, "Judgment," "Evidence," "What cannot be trusted," and "Remaining doubts").

Do not count the statement's values by hand; take them from the same computation as the earlier steps. Whether the time source has an outside reference is determined by what that time source's source is. For hosts that cannot be trusted, write their names so the reader knows what to leave out when reading.