Splice the snapshot and the increments wrong and the book is quietly wrong

In one line

Quotes arrive as one snapshot followed by incremental updates. You have to splice the sequence number the snapshot was taken at onto the increments' sequence numbers with an overlap, and if the sequence breaks even once, you must stop applying increments and wait for a new snapshot. An order book that misses this rule does not stop or go empty. It just keeps running, a little wrong.

Why this was needed

Sending the entire order book every time would exceed the bandwidth. So every market data feed uses the same shape. It takes one picture of the current state and sends it (the snapshot), and after that sends only the price levels that changed, one at a time (the increments). The receiver builds the order book from the snapshot and then applies the increments one after another.

The problem is the seam. Taking a snapshot takes time, and increments keep going out during that time. So a sequence number is stamped into the snapshot saying "everything up to here is already reflected." The receiver has to discard increments with sequence numbers earlier than the snapshot's and apply from there on. If you ignore this value and apply every increment you received, changes already reflected are applied one more time, and conversely if you discard every increment that came before the snapshot arrived, the changes in between disappear. This is why FIX-family specifications, when they deal with the resumption point, always carry "the last processed sequence number" along (FIX Trading Community standards documents; the meaning of the fields can be found at FIXimate).

How it works

There are only three actions in an increment: add, change, and delete. But most feeds do not send the action name. Quantity 0 means delete, and a nonzero value means change if that price is in the book and add if it is not. That is, the action is decided not by the message but by the state the receiver is holding. So even if you receive the same increment file, the result differs if the starting state differs. An implementation that stores quantity 0 as a value leaves a price level with quantity 0, and if that level takes the best quote position, the screen shows "0 shares waiting" as the best quote.

The order book has invariants that must hold. The best bid must be lower than the best ask. If the two are equal or reversed, it is crossed, and a crossed book does not exist in reality. If that state is on the screen, it means my reconstruction is wrong. Also, the quantity per price must be positive, and the sort order must always be maintained.

There is one more trap in the calculation that measures the invariants. The moment you subtract two prices to see whether the spread is one tick, binary floating point gives no answer. Subtracting 55.10 from 55.15 gives 0.04999999999999716, which is not equal to 0.05. Prices are decimal fractions, so they must be handled in decimal (decimal module documentation). Since this mistake raises no exception, it shows up as a quiet 0: "there was never a one-tick spread."

The last is resynchronization. A broken sequence means there is a change I did not receive, and there is no way to know what that change was. If you keep applying increments then, the book splits at the break and never comes back. The right handling is to stop applying increments and wait for the next snapshot. That is why the feed periodically re-broadcasts snapshots.

What it looks like in the field

One team got a report that "the best bid shows higher than the ask." Refreshing the screen made it look fine, so it was filed as a screen problem. In reality there had been a short gap a few minutes earlier, and the bid level that should have disappeared then had stayed. What the refresh fixed was not the screen but the order book. Orders that went out during those minutes had their prices set by looking at a quote that did not exist.

Another thing you often see is late arrival. If you apply in arrival order a line whose sequence number is in the past but that arrived later, a value already overwritten returns to its old value. This too passes without an error. The only defense is to apply in sequence-number order and discard sequence numbers that have already passed, not in arrival order.

The third is a somewhat different kind of incident. It happens when the reconstructed book is not held in just one place. If the screen and the strategy engine each apply increments, the two books diverge even though they receive the same input. That is because only one side detects the gap and resyncs while the other keeps applying. In a place like this, a human cannot tell by eye "which one is right." So the reconstruction should be done in one place and the result shared, or at the very least the two books should carry along the sequence number each has reflected up to. That is also the first thing you ask when investigating — up to which sequence number has this screen seen?

There is one more thing that must be decided together when fixing the receiving side. It is what to show as the order book while resynchronizing. If you leave the old book as it is, people believe it is the current quote, and if you empty it, the order screen looks frozen. Either way, the fact of that state must be visible on the screen and in the log. If you postpone this choice, then after you fix gap detection properly, reports that "the screen keeps going empty" actually increase.

What you will do in the next lab

You build a synthetic feed in which four symbols flow over one channel, build the order book from one snapshot, and apply about 1,300 increment lines. You add the rule that discards the overlapping sequence range, find the sequence number at the moment the book crosses, and count at how many price levels the result of continuing to apply at the break and the result of rebuilding from a new snapshot diverge. At the end you run the whole stream with an implementation that has both rules and confirm that it does not differ from the reference snapshot by even one level.