The Snack Order Was Saved, but the Warehouse Never Knew

Goal

Atomically save the work together with the send intent, and the receive ID together with the business effect, and recover after real HTTP resends and process termination.

Why it matters

If you conclude that work did not run merely because there is no response, you create duplicate application. Conversely, if you mark it complete before sending, you lose work that was never delivered. You implement the two boundaries separately with Python and SQLite, and you check the state that remains even after a process termination in which the cleanup code does not run.

It is estimated at 85 minutes. It is longer than the default session, so press +time to extend it before it expires. Files disappear after the session ends. Keep any code you need separately. You start knowing Python functions, exceptions, SQLite transactions, and the ACK recovery from the earlier module.

Data contract

All submitted code is /root/outbox/worker.py. The con that the functions below receive is an open connection with the correct role, and there is no transaction before the call. Functions other than open_store do not close the borrowed connection. After every success and failure, leave no open transaction. Always test source and sink as separate files, and do not connect to a production DB.

source schema:

CREATE TABLE orders (id TEXT PRIMARY KEY, qty INTEGER NOT NULL);
CREATE TABLE outbox (seq INTEGER PRIMARY KEY AUTOINCREMENT,
    id TEXT NOT NULL UNIQUE, qty INTEGER NOT NULL,
    sent INTEGER NOT NULL CHECK(sent IN (0,1)));

sink schema:

CREATE TABLE inbox (id TEXT PRIMARY KEY, qty INTEGER NOT NULL);
CREATE TABLE stock (id INTEGER PRIMARY KEY CHECK(id=1), total INTEGER NOT NULL);

The initial row of stock is (1,0). The initialization of each role is one transaction and preserves existing content. The checker creates and cleans up the temporary DB, the loopback HTTP server, and the child processes, so do not put a DB path or a fixed port in your code.

Steps

1. Build the contract for the send ID and quantity — In worker.py, implement the Exception subclasses Conflict and DeliveryError and validate_event(event). event is a dict with exactly the two keys id and qty. id is 1–64 characters of ASCII letters, digits, underscores, and hyphens, and qty is an int 1–1000 excluding bool. If invalid, it is a ValueError; if valid, it returns a new dict copy without changing the input.
2. Initialize two different stores — open_store(path, role) returns a sqlite3.Connection for the source or sink role. Open it with isolation_level=None and a busy timeout of 1 second and create the schema below only if it does not exist. Insert the sink's initial stock row (1,0) only if it does not exist. The initialization is one transaction, and on failure it closes the connection and propagates the error. It does not overwrite existing records with initial values. Any other role is a ValueError before connecting.
3. Save the order and the send intent together — enqueue(con,event,fault=None) handles it in BEGIN IMMEDIATE after validate_event. For a new ID it proceeds in the order orders insert, then fault('after-order'), then outbox(sent=0) insert, then fault('after-outbox'), then COMMIT, then fault('after-commit'), and returns True. Call fault only when it is given. An existing ID with the same qty returns False with no change, and a different qty is a Conflict. Errors before the commit are all rolled back and the original error is propagated, and errors after the commit do not erase the finalized records.
4. Read the unfinished batch in order — pending(con,limit) returns up to limit outbox rows with sent=0 in ascending seq order. Each element is a new dict with only id and qty. limit is an int 1–16 excluding bool, and any other value is a ValueError. The query does not change the DB or the input.
5. Limit the effect of a duplicate receipt to one — receive(con,event,fault=None) operates on a sink connection. After validation, in BEGIN IMMEDIATE, for a new ID it returns True in the order inbox insert, then fault('after-inbox'), then add qty to stock.total, then fault('after-total'), then COMMIT, then fault('after-commit'). The same ID and qty is False, and the same ID with a different qty is a Conflict. A different ID with the same qty is separate work. Errors before the commit roll back everything, errors after the commit preserve the finalized state, and the original error is propagated.
6. Leave the completion marker only on confirmed items — mark_sent(con,event) changes to 1 only the sent of the source outbox row that matches the validated id and qty. An unknown ID is a ValueError, a different qty is a Conflict, and on failure it does not change the DB state. It returns True for a new marker and False if already complete. It does not delete orders and outbox rows and leaves no transaction.
7. Mark after the ACK and stop at the first failure — dispatch(con,send,limit=8,fault=None) processes only a finite batch of pending, in order. It gives send a copy of the event and, after receiving an exact True ACK, calls fault('after-delivery') and then mark_sent. False, None, 1, or a string is a DeliveryError, and exceptions from send or fault are propagated as they are. It stops at the first failure and preserves the earlier completions. The return value is the number marked complete this time. It does not hold a write transaction during the send, and even if the callback changes the argument, it marks complete only the item it originally selected.
8. Recover with HTTP and a real process restart — Implement send_http(url,event,timeout=1). After validate_event, send an HTTP POST with JSON, and return True only when the status is 200, the JSON dict of at most 1024 bytes has exactly the two keys id and accepted, the id equals the request's, and accepted is True. A wrong ACK is a DeliveryError; HTTP and connection errors are propagated, and the response is closed. timeout is a positive finite int/float excluding bool and at most 5 seconds, and anything else is a ValueError. Allow only URLs of the form http://127.0.0.1:포트/events만 (the placeholder is the port number), with an explicit port 1–65535 and no userinfo, query, or fragment. Do not use an automatic proxy or redirects. The final check actually terminates the publisher after the receive commit and verifies that a new process resends the same ID and increases the stock only once.

Notes

• Final diagnostic command: python3 -B /opt/fixtures/outbox/check.py 8 /root/outbox/worker.py. For earlier steps, replace 8 with that step's number. It runs against a temporary store without modifying the submission file, and grading is limited to 12 seconds.
• You use only the standard library, and no internet, installation, or extra capability is needed. The provided HTTP server is loopback-only and is a teaching server without authentication or TLS. The timeout of send_http is a wait limit per blocking socket operation, not a total deadline for the whole request.
• It verifies the finite batch of a single delivery loop and a process restart on the same disk. It does not verify an automatic retry scheduler, multi-worker claim/lease, disk loss, atomicity of external payments, or large-scale performance.
• Even if the response vanishes after the receive commit, do not delete the existing work. Deliver it again with the same ID and content, and confirm from an independent connection that the effect happened once.

Build the contract for the send ID and quantity

In worker.py, implement the Exception subclasses Conflict and DeliveryError and validate_event(event). event is a dict with exactly the two keys id and qty. id is 1–64 characters of ASCII letters, digits, underscores, and hyphens, and qty is an int 1–1000 excluding bool. If invalid, it is a ValueError; if valid, it returns a new dict copy without changing the input.

That int(True) is 1 and that the quantity is valid under the contract are different things. Check the whole string with the regular expression.

Initialize two different stores

open_store(path, role) returns a sqlite3.Connection for the source or sink role. Open it with isolation_level=None and a busy timeout of 1 second and create the schema below only if it does not exist. Insert the sink's initial stock row (1,0) only if it does not exist. The initialization is one transaction, and on failure it closes the connection and propagates the error. It does not overwrite existing records with initial values. Any other role is a ValueError before connecting.

CREATE TABLE IF NOT EXISTS and INSERT OR IGNORE play different roles. On success, leave no open transaction.

Save the order and the send intent together

enqueue(con,event,fault=None) handles it in BEGIN IMMEDIATE after validate_event. For a new ID it proceeds in the order orders insert, then fault('after-order'), then outbox(sent=0) insert, then fault('after-outbox'), then COMMIT, then fault('after-commit'), and returns True. Call fault only when it is given. An existing ID with the same qty returns False with no change, and a different qty is a Conflict. Errors before the commit are all rolled back and the original error is propagated, and errors after the commit do not erase the finalized records.

If you put a COMMIT between the two INSERTs, only the order remains when the process exits. You have to keep the fault points to test the intermediate states.

Read the unfinished batch in order

pending(con,limit) returns up to limit outbox rows with sent=0 in ascending seq order. Each element is a new dict with only id and qty. limit is an int 1–16 excluding bool, and any other value is a ValueError. The query does not change the DB or the input.

ID z can be accepted before a. Use the stored insertion order instead of the alphabetical order of the business ID.

Limit the effect of a duplicate receipt to one

receive(con,event,fault=None) operates on a sink connection. After validation, in BEGIN IMMEDIATE, for a new ID it returns True in the order inbox insert, then fault('after-inbox'), then add qty to stock.total, then fault('after-total'), then COMMIT, then fault('after-commit'). The same ID and qty is False, and the same ID with a different qty is a Conflict. A different ID with the same qty is separate work. Errors before the commit roll back everything, errors after the commit preserve the finalized state, and the original error is propagated.

Do not commit the receive record and the business effect separately. False is not a failure; it is a valid duplicate for which no new effect was applied this time.

Leave the completion marker only on confirmed items

mark_sent(con,event) changes to 1 only the sent of the source outbox row that matches the validated id and qty. An unknown ID is a ValueError, a different qty is a Conflict, and on failure it does not change the DB state. It returns True for a new marker and False if already complete. It does not delete orders and outbox rows and leaves no transaction.

Preserve the work and the intent for later reconciliation. Do not complete the whole batch without a WHERE id condition.

Mark after the ACK and stop at the first failure

dispatch(con,send,limit=8,fault=None) processes only a finite batch of pending, in order. It gives send a copy of the event and, after receiving an exact True ACK, calls fault('after-delivery') and then mark_sent. False, None, 1, or a string is a DeliveryError, and exceptions from send or fault are propagated as they are. It stops at the first failure and preserves the earlier completions. The return value is the number marked complete this time. It does not hold a write transaction during the send, and even if the callback changes the argument, it marks complete only the item it originally selected.

If you mark complete first, you lose an unfinished item when the process exits before the send. Do not confuse receive's False with an HTTP ACK failure.

Recover with HTTP and a real process restart

Implement send_http(url,event,timeout=1). After validate_event, send an HTTP POST with JSON, and return True only when the status is 200, the JSON dict of at most 1024 bytes has exactly the two keys id and accepted, the id equals the request's, and accepted is True. A wrong ACK is a DeliveryError; HTTP and connection errors are propagated, and the response is closed. timeout is a positive finite int/float excluding bool and at most 5 seconds, and anything else is a ValueError. Allow only URLs of the form http://127.0.0.1:포트/events만 (the placeholder is the port number), with an explicit port 1–65535 and no userinfo, query, or fragment. Do not use an automatic proxy or redirects. The final check actually terminates the publisher after the receive commit and verifies that a new process resends the same ID and increases the stock only once.

Check ProxyHandler({}) and HTTPRedirectHandler of urllib.request. Even if the request arrives twice, the warehouse's effect must happen once.