Explain Changes with Evidence

In one line

A change report has to reconcile the approval, the execution audit, and the observed current rows at the same point in time, and must not mix facts you could not confirm into the word "complete".

Why this was needed

You received a message that the alien festival's cancellation job has finished. The operations owner says "We processed three" and the customer asks "Did all three go back to the way they were?" The same number, three, is hiding different questions. Whether the three approved IDs are right, whether the change was committed at that time, whether another owner changed it afterward, and whether it is still in that state now are all different.

Would it solve the problem if you wrote a longer report? Stretching an unsupported completion sentence to three pages only makes a more plausible misunderstanding. This time you are not the person who fixes the DB again but the owner who investigates a change that has already been executed and explains it so that the customer can decide the next action. The inputs are the approval list, the audit at execution time, and the current rows at the observation time. The output is a small evidence bundle that has both a structure that a machine can check and an explanation that a person can read.

The earlier chunks lesson continued running the job. This reporting tool is not entrusted with execution authority. If a mismatch is found, it writes the reason for the hold, and does not overwrite the approval or the current values to fit the report. The moment you change the facts to make the report green, the boundary between investigation and execution disappears.

How it works

1. Match the sets and the meaning before the count

If the approved IDs are 1, 2, and 5 and the audit IDs are 1, 2, and 31, both are three items but it is not complete. ID 31, which is outside the approval, is an incident candidate to check, and ID 5 is a target with no audit. Comparing the lengths of the two sets alone cannot discover this difference.

The ID is not enough either. Only when the audit's previous revision, later revision, and quantity match the approved contents do you count it as the commit evidence for that change. The later revision the audit recorded must be the approved revision+1. The current row's customer, quantity, state, and revision are separate material for checking whether it changed again afterward. Even in the same cancelled state, a larger revision may mean later work, so you do not quietly treat it as matching.

The classification in this customer contract is as follows. committed is an ID that has an audit that exactly matches the approval, and remaining is the rest of the approved IDs. Among committed, if the current values also match it is matching, if the current row is missing it is missing, and otherwise it is drifted. An audit outside the approval, or an audit whose before and after versions or quantity differ, is left separately as invalid_audit. Data whose format itself is broken or that has duplicate IDs is rejected before analysis.

From the current state of a remaining ID with no audit alone, you cannot assert that it was not executed. There may have been work through another path, or the audit may have been omitted. This lesson reports it as "there is no commit evidence". To start the actual execution again, you need the current condition and the new approval verification from the previous lesson. Do not use the report as an execution permit.

2. Make several SELECTs see the same moment

After reading the approval, another connection commits the orders and the audit together. If the reporting tool then reads the audit and the orders, the first query may be at the old point in time and the next query at the new one. The fact that you put in BEGIN alone does not make several queries the same moment.

This collection uses a PostgreSQL REPEATABLE READ READ ONLY transaction. It keeps ordinary table reads within the same observation as a consistent snapshot while preventing the reporting code from modifying the business tables. In the isolation levels official documentation, compare the read time points of Read Committed and Repeatable Read, and also read the READ ONLY restriction of SET TRANSACTION.

The lab does not look only at whether you wrote the isolation level as a string. At the after-plan or after-audit point, another connection actually commits real orders and audit. The new change must not be mixed into this collection, and must appear only when you observe again after the collection is finished. It also checks whether the real DB rejects an attempt to UPDATE within the read-only section.

This guarantee is not a guarantee that all business data is correct. If the original audit was damaged, that damage appears as it is even in a consistent snapshot. It also does not bind the external APIs of multiple systems to the same moment. The target this time is ordinary tables inside the same PostgreSQL, and the business consistency of the collected data is checked separately in the next step.

3. Return the state of the borrowed connection

A psycopg connection can be reused. If in this call you set read-only, the isolation level, and a short time limit, you must not leave them as they are for the next call. This lab takes as a contract a connection that is autocommit=True and has no open transaction at the start of a call. It applies the settings only inside the collection context and returns to IDLE after success or failure.

You do not close the borrowed connection; you close only the connection that publish created itself. There is also no reason to keep the DB snapshot open while writing the file. After collecting the data, you close the connection and then proceed with serialization and the file replacement. See psycopg transaction management and recheck the difference between a nested transaction and a real commit. Separately from the documentation version, the lab runs on the installed psycopg 3.2.3.

4. Build the explanation shown to people from the same evidence

If you write hold in the machine-facing result and write "We completed it" in the customer explanation, the material already conflicts internally. This bundle includes evidence, report, and summary together. The summary is plain text that, from the same analyze result, shows the approved, committed, unconfirmed, currently matching, later-diverged, missing, and audit-mismatch IDs.

The word "complete" also gets a scope. complete means "complete at the observation time" and is not a promise that the state is kept after the observation. If there is a problematic audit, a later difference, or a missing row, it is hold, and even with no such problem, if an approved ID with no commit evidence remains, it is incomplete. The reason hold comes before complete is that you have to explain the present anomaly before executing the rest.

The report is not SQL or HTML. The strings and numbers in the JSON are treated as data and are not executed. The timestamp and snapshot notation are metadata that identify later which observation is being referred to. You must not forget that a value can be typed in by hand and explain it like an officially certified timestamp.

What it looks like in the field

A hash can match and the report can still be false

If you fix the key order, whitespace, and UTF-8 representation when serializing, the bytes and the hash of the same canonical data become the same. This is a local rule of this format, and we do not claim it is a standard in which all JSON tools automatically produce the same bytes. You distinguish numbers, strings, and booleans, and you also reject inputs such as duplicate keys and NaN. See the serialization options and object_pairs_hook in the Python JSON documentation.

A hash is a means of checking whether the contents changed, not a signature that authenticates the origin. If someone changes only the verdict to complete and recomputes the hash, it can pass a simple hash check. That is why you recompute report and summary from evidence and compare them together. Conversely, if someone consistently builds all of the evidence anew, including the observation time, and also computes the hash, this check alone cannot tell the fiction apart. The lab also shows a counterexample in which this case passes. Internal consistency and authenticity are different requirements.

When you deliver to a real customer, a trustworthy collection path, access control, and an immutable record and approval system may additionally be needed. Having built this file format does not mean you have implemented the whole evidence retention policy of that organization. User authentication and per-customer DB access permissions are also not the role of this analysis function.

If it fails after overwriting the report file only halfway

If you first open the existing report.json in write mode, a later error can erase even the earlier finished version. You write the finished bytes to a separate temporary file in the same directory, close it, and then switch the destination with os.replace. If the error is before the replacement, the old file remains, and if only the response was lost after the replacement, the new file remains in a finished state. Read the os.replace documentation together with where the failure happens.

The lab also checks whether, on an ordinary exception, it cleans up its own temporary file and does not touch other files. On a forced process kill, finally may not run, and on a power failure the filesystem durability problem is added. This verification covers the preservation in the file replacement under a normal run and an injected exception, and it does not prove a forced kill, disk corruption, or a power failure.

An FDE's report helps the next decision

The Palantir FDSE job posting covers customer-specific implementation and collaboration with multiple stakeholders. On that basis we designed an assignment that builds together an explanation the customer will read and a structure the technical team will re-verify. It does not mean that we reproduced the company's internal forms or required technical patterns.

What you will do in the next lab

You collect the change evidence of the space festival consistently, and separate the same count with a different ID, audit version errors, and later changes. You build the explanation that people read and the machine verdict from the same evidence, and you also reject a wrong conclusion whose hash was recomputed and duplicate JSON keys. At the end, you connect everything from read-only DB collection to publishing that preserves the existing file.

The lab uses only fictional data and a per-student disposable DB. There is no task that sends a report to an external customer or modifies a production DB.