Remember Duplicates Outside the Pod

In one line

To make retries safe, you distinguish the execution ID from the business ID, and at the boundary where the side effect is stored, you must handle the same business key consistently.

Why this was needed

Suppose you changed the shipping program to create a random UUID at startup and check whether the request was already processed. The probability of a UUID collision is small, but duplicate shipments can still happen. That is because every new Pod creates a new UUID. A retry of the same order looks like different work. The property that an identifier is unique and the property that it recognizes the same work as the same are different.

Writing "processing complete" to a temporary file inside the Pod may also fail to cross the retry boundary. When a Job creates a new Pod, you cannot assume the memory or local files of the previous run come along. The lab does not hide this difference: it records the Pod UID on each request, but uses a separate business key to decide whether a shipment is a duplicate.

How it works

A request in the lab has four values. case_id is an independent experiment scope for the work, and work_id is the logical order within that scope. pod_uid tracks the run that sent the request, and mode distinguishes the comparison group from idempotent handling. When you rerun the same work, case_id and work_id are kept, and the new Pod UID must be different. In a real service, you have to decide separately on the key scope, including the account, tenant, and kind of work, and on the reuse policy.

The ledger manages request attempts and shipment side effects in different tables. Even if a request comes in twice, the shipment result can be created only once. When the same key is received again, it returns the first shipment number and keeps recording a new request attempt. Thanks to this, removing the duplicate does not also erase the observation information.

The core is the race between the lookup and the store. If two requests arrive almost at the same time and each reads "none" and then stores a new shipment, a duplicate can arise. The lab ledger handles the key lookup and the insert inside a SQLite write transaction, and puts a uniqueness constraint on the idempotency key. A single if statement on the client does not stand in for concurrency safety. If an error occurs, that transaction is not reported as a successful shipment.

실행 추적: Pod A → 요청 1
          Pod B → 요청 2
업무 판정: 같은 업무 키 → 같은 배송 번호

This is not called an arbitrary exactly-once delivery guarantee. The example merges the insertion of one kind of row into one inside the ledger. A real carrier, mail server, or other DB does not automatically take part in the ledger's transaction. Additional contracts such as an idempotent API of the external system, result lookup, an outbox, and reconciliation are sometimes needed. You have to be able to explain which storage boundary you protected.

What it looks like in the field

An operator may create a new Job with a different name from the completed Job and rerun the same order. If you use the Job UID as the idempotency key, the new run gets a new key and so does not recognize the previous side effect. Conversely, if you keep the business key, the same result can be reused even though the Kubernetes object is different. This is why deleting the execution object and deleting the business processing history are different lifecycles.

Tying all requests to the same key is also wrong. If even different orders are merged into the same shipment, duplicates decrease but legitimate work is lost. The test must check not only repetitions of the same key but also that different keys are each processed. Whether to allow or reject a case where the key is the same but the body differs, and how long to keep keys, are also part of the real contract. The small request model in this lab has no address or items, so it does not claim to have implemented such a conflict policy.

A CronJob creates Jobs at set times, and it does not take over the role of the idempotency ledger. Even if you set concurrencyPolicy to Forbid, it is a setting that limits the overlap of Jobs created by the same CronJob, not a duplicate-handling contract for the external ledger. You have to check separately whether the business key carries through to other CronJobs, manual Jobs, and the operator's rerun paths.

Also distinguish how long the storage survives. The experiment's ledger is in SQLite inside an emptyDir of a separate Pod. It is not affected by replacing work Pods, but if the ledger Pod itself is replaced, the data disappears. So check that the ledger Pod UID and the process boot_id have not changed. This setup does not solve permanent retention, multi-node high availability, or disaster recovery.

What you will do in the next lab

With the same failure after the store kept in place, turn on the ledger's idempotent handling. Check two requests and one shipment together, and see whether the shipment number is preserved even when the same work is rerun with a different Job. At the end, do not erase the ledger attempt list or read only the success marks; explain whether different work remains and only the duplicate work was merged.

Official documentation and scope

This reading connects the execution characteristics of Kubernetes with an application's storage contract. It does not reduce idempotency to the existence of a single file or a Job's success condition.