Stop an old worker from overwriting a new result

Goal

You implement an ownership lease and an increasing fencing token in SQLite.

Why it matters

Worker A took a job and then stalled. When the lease expired, B took the same job and completed it, but A, which came back to life late, also wrote a completion result. Deciding the lease duration alone cannot block the writes of an old worker. At the time of storing, you must also check the owner and the generation number.

Steps

1. In /root/work/idem-fencing-lab/service.py, init_db(path) idempotently creates jobs(id TEXT PRIMARY KEY, owner TEXT, until REAL NOT NULL DEFAULT 0, fence INTEGER NOT NULL DEFAULT 0, result TEXT, done INTEGER NOT NULL DEFAULT 0).

Prepare it once at the beginning. It does not overwrite an existing file.

mkdir -p /root/work/idem-fencing-lab
test -e /root/work/idem-fencing-lab/service.py || cp /opt/fixtures/ten_labs/idem-fencing-lab/service.py /root/work/idem-fencing-lab/service.py
cd /root/work/idem-fencing-lab

1. In /root/work/idem-fencing-lab/service.py, enqueue(path, job_id) inserts an id that does not exist in the default state and returns True, and if it already exists it does not change the state and returns False.

2. In /root/work/idem-fencing-lab/service.py, state(path, job_id) returns the row as a dict with the keys id, owner, until, fence, result, and done, and None if it does not exist.

3. In /root/work/idem-fencing-lab/service.py, claim(path, job_id, owner, now, ttl) is a ValueError if ttl<=0. A job that does not exist, a completed job, or a job with until>now gives None. Otherwise it stores owner and until=now+ttl, increments fence by 1, and returns the new fence.

4. In /root/work/idem-fencing-lab/service.py, renew(path, job_id, owner, fence, now, ttl) is a ValueError if ttl<=0. Only when owner and fence match, done=0, and until>now does it change until=now+ttl and return True; otherwise it returns False.

5. In /root/work/idem-fencing-lab/service.py, complete(path, job_id, owner, fence, now, result) stores result and done=1 and returns True only when it is the current lease (owner and fence match, done=0, until>now). Otherwise it returns False and preserves the existing result.

6. In /root/work/idem-fencing-lab/service.py, release(path, job_id, owner, fence) changes the row where owner and fence are the same and done=0 to owner=NULL and until=0, and returns True. fence is preserved. Otherwise it returns False.

7. In /root/work/idem-fencing-lab/service.py, claim_many(path, job_id, owners, now, ttl) calls the claim of each owner concurrently with ThreadPoolExecutor and returns a list of return values in input order. For a new job, exactly one gets a fence and the rest must be None.

Notes

• Do it in the existing lab-dev environment, without the internet or package installation.
• Each step runs within a 45-second grading budget. Do not add real sleep or network calls.
• Grading freshly imports the submitted module and checks it with independent inputs and a temporary DB. Implement the contract instead of returning expected values as constants.
• FastAPI official documentation · pytest official documentation · Python sqlite3
• Limitation: the clock is a non-decreasing number given by the caller, and clock synchronization between several servers is not modeled. The external side effects of the business logic are not fenced automatically. Systems outside the storage also need the same token verification or separate idempotency handling. SQLite uses real locks and transactions, but it does not represent the throughput of a large distributed queue.

Create the lease state table

In /root/work/idem-fencing-lab/service.py, init_db(path) idempotently creates jobs(id TEXT PRIMARY KEY, owner TEXT, until REAL NOT NULL DEFAULT 0, fence INTEGER NOT NULL DEFAULT 0, result TEXT, done INTEGER NOT NULL DEFAULT 0).

Prepare it once at the beginning. It does not overwrite an existing file.

mkdir -p /root/work/idem-fencing-lab
test -e /root/work/idem-fencing-lab/service.py || cp /opt/fixtures/ten_labs/idem-fencing-lab/service.py /root/work/idem-fencing-lab/service.py
cd /root/work/idem-fencing-lab

If the generation number is reset when a worker restarts, old tokens become valid again.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/01-contract.sh.

Register only the first job

In /root/work/idem-fencing-lab/service.py, enqueue(path, job_id) inserts an id that does not exist in the default state and returns True, and if it already exists it does not change the state and returns False.

Use INSERT OR IGNORE so that re-registration does not reset a lease in progress.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/02-contract.sh.

Read the current state

In /root/work/idem-fencing-lab/service.py, state(path, job_id) returns the row as a dict with the keys id, owner, until, fence, result, and done, and None if it does not exist.

Read the stored state over a separate DB connection to distinguish it from a cache inside the process.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/03-contract.sh.

Take an expired lease atomically

In /root/work/idem-fencing-lab/service.py, claim(path, job_id, owner, now, ttl) is a ValueError if ttl<=0. A job that does not exist, a completed job, or a job with until>now gives None. Otherwise it stores owner and until=now+ttl, increments fence by 1, and returns the new fence.

Do the read and the update inside a single BEGIN IMMEDIATE. The boundary now==until can be reassigned.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/04-contract.sh.

Only the current generation renews the lease

In /root/work/idem-fencing-lab/service.py, renew(path, job_id, owner, fence, now, ttl) is a ValueError if ttl<=0. Only when owner and fence match, done=0, and until>now does it change until=now+ttl and return True; otherwise it returns False.

If you revive an already expired ownership with renew, it conflicts with the new worker.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/05-contract.sh.

Reject a stale completion

In /root/work/idem-fencing-lab/service.py, complete(path, job_id, owner, fence, now, result) stores result and done=1 and returns True only when it is the current lease (owner and fence match, done=0, until>now). Otherwise it returns False and preserves the existing result.

The completion write must also have the lease check to block a worker that returns late.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/06-contract.sh.

Only the current worker returns the lease

In /root/work/idem-fencing-lab/service.py, release(path, job_id, owner, fence) changes the row where owner and fence are the same and done=0 to owner=NULL and until=0, and returns True. fence is preserved. Otherwise it returns False.

If you reset even fence when returning the lease, past token numbers get reused.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/07-contract.sh.

There is one winner among concurrent claims

In /root/work/idem-fencing-lab/service.py, claim_many(path, job_id, owners, now, ttl) calls the claim of each owner concurrently with ThreadPoolExecutor and returns a list of return values in input order. For a new job, exactly one gets a fence and the rest must be None.

Do not close the connection after a pre-lookup and then update. The lock must protect both operations together.

After saving, check with bash /opt/lab/checks/idem-fencing-lab/08-contract.sh.