An order's status cannot be just any value

In one line

An order report is not a free-form record but the output of a state machine, and the quantities in it obey equations that must not break. Most of "what went wrong?" in an incident investigation comes out when you check these two by machine.

Why this was needed

An order has anywhere from three or four to dozens of reports attached. Reading those lines by eye to find anomalies is only possible for the first few orders. A day's worth is tens of thousands of lines, and only five or six of them are a problem. So the first thing an FDE does on receiving customer data is not to read but to filter.

The reason a filter can be built is that this domain has rules. The FIX protocol expresses an order's state in a field called OrdStatus and fixes names such as New, PartiallyFilled, Filled, and Canceled. The meaning of the fields can be checked at Fiximate. Which states can lead to which differs slightly by market and broker (in this lab the transition table is given as data and taken as this lab's assumption), so what matters is not memorizing the table but that a table exists and getting code to read that table.

How it works

There are five lines of checking.

First, illegal transitions. If the path from the earlier report's state to the later report's state is not in the table, that is a problem. The most important class here is terminal states. If something comes out again from a state with nowhere further to go — filled, canceled, rejected, expired — it is either a late-arriving report or a real defect. But sometimes the table itself is wrong. If a state is declared terminal and yet one outgoing transition is left, the checker lets that transition through as normal. So the moment you read the table, you look for the table's own contradictions first.

Second, the quantity equation. For a live order, CumQty + LeavesQty == OrderQty. It means the cumulative fills plus the remaining quantity equal the order quantity. The thing to watch is that once the order is finished, this equation does not hold. If you cancel a half-filled order, the remaining half disappears and the leaves quantity becomes 0. A checker that applies the equation unconditionally raises every normal cancel as a false positive.

Third, monotonicity of the cumulative. CumQty does not decrease. If it looks like it decreased, reports arrived out of order or a rewind happened.

Fourth, overfill. If the sum of executed quantities exceeds the order quantity, that is an incident with money on it. But if the same report comes in twice, the sum inflates and looks like an overfill. So this judgment has to be redone after deduplication.

Fifth, average price. AvgPx must be the quantity-weighted average of the fills up to that point. If you use floating point here, the last digit wobbles and perfectly good data comes out looking off. Do the money calculation with decimal and fix the rounding mode.

And one last question remains. Among what you filtered out, what is a real defect? If the same report came in twice, removing duplicates makes the anomaly disappear. If the arrival order got swapped, re-sorting by exchange time makes it disappear. What remains after both is the real thing. If you report "30 anomalies" without separating these three lines, the other side can do nothing.

What it looks like in the field

Once, overfill alerts were coming in at twenty a day. They all had the same cause: retransmitted fill reports were being summed without deduplication. There was no real overfill among those twenty, and the one truly dangerous case was found weeks later through another route. When a filter is full of noise, nobody looks at its alerts anymore.

Another was the equation check. A checker that raised every canceled order as an anomaly sent out thousands of alerts a day, and in the end that check was switched off entirely. That is the result of looking only at the quantity and not at the state.

The third was the transition table itself. In the document the customer gave us, filled was written as a terminal state, but in the table the code read, one outgoing line was left from that state. Where the document and the code disagree, what the checker sees wins. So when you receive a new customer's data, you check the transition table against itself first. Is there an exit from a state declared terminal? Is there a state that cannot be reached from any state? Is there really a path from the start state to a terminal one? These three can be checked in a few lines from the data alone, and if any of them catches something, you cannot trust any of the later judgments.

Finally, one practical addition. It is better to build these checks in a form you can hand over to the customer. If you use them once for the investigation and throw them away, the same problem comes back three months later and you write it from scratch again. If the rules are in data and the code reads that data, the same tool runs as it is even when the market or the broker changes.

What you will do in the next lab

You build a transition table and execution reports for twelve orders, and start by finding the table's own contradictions. Then you check, in order, illegal transitions, the quantity equation, cumulative regressions, overfills, and average price, and at the end you split what is explained by duplicate reports and out-of-order arrival from what is not, and produce a summary by cause. Orders that ended in cancel and orders that ended in reject are mixed in, so if you apply the equation unconditionally you get stuck right there.