A Message Can Arrive Twice

In one line

No response and not processed are different facts, but the sender cannot tell them apart, and so retransmission creates a duplicate transfer — the only way to prevent it is an idempotency key.

Why this was needed

The messages banks exchange with each other are called messages. One account transfer is one message, and each message carries a message number.

The problem is this. You sent a message, but no response came. What the sender knows at this point is only one fact: "I did not receive a response." The other side may not have received it, or it may have received and processed it and the response was lost. The sender has no way to tell these two apart.

From the viewpoint of the side sending the money, if it does not retransmit, the customer's transfer ends in failure. So it retransmits. If the other side has already processed the first message, the same money now goes out twice.

How it works

The solution is an idempotency key. It is an identifier attached one per original transaction.

전문 번호(msg_id)    전송할 때마다 새로 매긴다. 재전송하면 다른 번호가 붙는다.
멱등키(idem_key)     원거래 하나에 하나. 재전송해도 같은 값을 쓴다.

The receiving side remembers idempotency keys, and when a key it has already seen comes again, it does not process it and returns the result of the first processing as it is. The point is to give the success response again rather than answer with a failure. If it answers with a failure, the sender retransmits again.

An idempotency key must have an expiry. This is because you cannot remember forever. It is usually set somewhere between a few days and a month, and if this period is shorter than the retransmission window, idempotency breaks silently.

Here is a decisive fact an FDE must know. A duplicate transfer is not caught by reconciliation.

If we sent the message twice and the other side processed it twice, our ledger also holds 2 items, and the other side's settlement file also contains 2 items. The count and the amount match exactly. The total reconciliation passes perfectly.

There is only one place it gets caught. It is to look at whether two or more messages are attached to the same idempotency key. If reconciliation asks "are the two books the same," this query asks "within our own books, did the same transaction go out several times?" The two questions are entirely different.

What it looks like in the field

First, you must not rebut a "the transfer went out twice" inquiry with the reconciliation result. That reconciliation matched is no evidence at all. Until you query grouped by idempotency key, you cannot say there is no duplicate transfer.

Second, a short timeout value increases this incident. On a day when the other system's processing time grows — the close batch overlapped, or just before a holiday — timeouts occur in bulk and retransmissions pour in. That is why duplicate transfers do not occur sporadically but cluster on particular dates. If you look at the time distribution first during an investigation, the cause narrows quickly.

Third, the source of a retransmission can be a person. The case where no response comes on the screen and an operator presses the button once more, the case where a batch judges it a failure and throws it again, and the case where a monitoring tool retries all produce the same result. Without an idempotency key, you would have to block all three, which is impossible.

Fourth, you handle the return of an excess amount as a correction, not as a new transfer. If you just send the extra money that went out in the opposite direction, three transfers remain in the ledger, and you cannot tell which is the original transaction and which is the correction. Only if you send it with a reversing entry on the ledger side and a return message carrying the original-transaction reference on the external side can it be traced later.

What you will do in the next lab

After reconciling our ledger against the counterparty settlement file, you find duplicate transfers by idempotency key in a state where the reconciliation passes perfectly. You compute that 5 messages are attached to 2 idempotency keys and how much was sent in excess. And you confirm that this amount is mixed as it is into the net settlement position — that it must be adjusted before settlement.