A ledger is not corrected by erasing

In one line

An execution that has already gone out is not fixed by erasing it as wrong but by adding a record that reverses it. It takes more space, but even if someone asks months later, you can read out "why did this number come to be this?"

Why this was needed

An execution is not the end. If the price was entered wrongly or the counterparty does not confirm, that execution is canceled, and if the values are only slightly off, it is amended. The problem is what comes next. The execution ledger has already flowed out to many places. Position reports went out, risk limit calculations ran, and P&L was stamped. If you quietly rewrite one line of the ledger in that state, yesterday's reports and today's ledger go out of line, and the reason for the mismatch is nowhere in the ledger.

That is why financial books write afterward. A cancel leaves the original record in place and adds one offsetting record with the opposite sign, and an amendment offsets the original record and then writes one line again with the corrected values. It is a long-standing method in accounting, and the execution reports of the FIX protocol also have separate places to announce cancels and amendments. The meaning of the fields can be checked at Fiximate. Concrete procedures such as how many minutes you have to file a cancel differ by market, so in this lab the rules file given as data is taken as the lab's assumption.

How it works

First, check what the notice points to. A cancel or amendment notice contains the identifier of the original execution, and if that identifier is not in the ledger, you must do nothing. The implementation that "finds a similar execution and applies it" is the most dangerous here. If you reverse the wrong execution, one wrong case becomes two.

Next is the case of receiving the same notice twice. Retransmission is normal. If you do not remember the notice identifier and offset whatever you receive, an execution that was canceled once gets offset twice and the position flips to the opposite side. That the result must be the same however many times you feed in the same input is idempotency.

Third is a notice that comes again for an already-canceled execution. You must not accept a re-cancel, or a price amendment for a canceled execution. For that, the processor has to hold "what have I canceled so far."

An amendment takes more work than a cancel. Only the price may change, or only the quantity, so values not in the notice must use the original values as they are, and depending on the changed values, the execution amount, the average price, and the position all move together. Handle amounts with decimal and fix the rounding mode. With floating point, one amendment makes the last digit wobble, and places you did not fix look as if they are off.

The last is a late cancel. A notice that arrives after the position report has gone out is not finished by fixing only the books. The report that already went out becomes wrong, so it becomes a target for re-reporting. But not every late notice moves the position either. An amendment that changes only the price does not touch the quantity, so the position stays the same and only the amount changes. If you do not separate these two, the re-report list becomes uselessly long, and a long list is one nobody looks at.

What it looks like in the field

Once, a batch read a cancel notice that arrived overnight twice, and the position was stamped with the opposite sign all day. The cause was one thing: the notice identifier was not recorded. Worse, that batch was overwriting the original records, so it took two days to find out what disappeared and when.

Another was a position reconciliation. The recomputed positions and the reports that had already gone out differed slightly by symbol, and most of the difference was explained by late-arriving cancels, but one symbol was not explained. That one unexplained symbol was the real defect. If the differences had been lumped together and written as "reconciliation mismatch," that one case would have been buried.

The third thing you often meet is a ledger that cannot be traced. Even if you do the offsetting and rebooking correctly, if you do not write down which notice that line came from, nobody can explain it weeks later. The question that comes up then is always the same — why did this symbol's quantity come to be this? If you attach to each line of the ledger the identifier and reason of the notice that created it, that question ends in one lookup. There is one thing to be careful about when attaching the reason. A retransmitted copy also has a reason attached, but what must be left in the ledger is the reason of the notice that was actually applied. An implementation that picks the last one from the notice stream is quietly wrong here.

And all of this handling must not depend on order. Notices may not arrive in the original order, and a batch may run twice. If you judge by holding as state what you have applied so far, the same ledger comes out however many times and in whatever order you feed the same input. This property is not to be asserted in words but tested by feeding the same input through twice and comparing the result files. Without that test, the sentence "it is idempotent" is only hope.

What you will do in the next lab

You build an execution ledger, a stream of cancel and amendment notices, and a position report that has already gone out. After attaching the notices to the original executions and separating what to apply from what to discard, you build a new ledger by adding offsetting records while leaving the original records as they are. You reflect the amendments to recompute the execution amounts and average prices, then recompute the positions and compare them with the existing report. You confirm from files that the ledger is the same even when the same notice stream is fed in twice, and gather the notices that arrived after the report time to produce a re-report target list and a correction report.