Building a Job Submission and Status Polling API

Goal

Implement the full contract of an asynchronous API that starts with 202 Accepted — acceptance, status lookup, polling regulation, idempotent submission, and completion notification.

Why it matters

If you make a 30-second job a synchronous API, three things break at once. It hits the load balancer's idle timeout, client retries run the same job twice, and workers are tied up for 30 seconds each. An asynchronous API splits this into acceptance and checking. But surprisingly many APIs stop after returning only 202. The client does not know where to look for the result, does not know how long to wait, and does not know whether it is safe to retry. This lab fills in the missing pieces of that contract one by one — the Location header, the status vocabulary, Retry-After, the idempotency key, and the webhook. Once you have all of these, the client can retry with confidence.

Steps

1. Start /root/aj/api.py on 127.0.0.1:8140. POST /jobs receives {"n":5} and puts it in the queue q:jobs2.
2. The POST /jobs response is 202 with the Location: /jobs/<job_id> header and the body {"job_id":"...","status":"queued"}.
3. GET /jobs/<job_id> returns 200 and a status. The value is one of queued, running, succeeded, and failed. A nonexistent id returns 404.
4. /root/aj/worker.py takes an item out of the queue, changes its state to running, and after processing changes it to succeeded.
5. The response for looking up a completed job must have a result key. In the queued state there must be no result key.
6. A queued or running response carries the Retry-After header as integer seconds, and a succeeded response does not.
7. If you submit twice with the same Idempotency-Key, it returns the same job_id and only 1 item goes into the queue.
8. Start /opt/app/hooksink.py on 127.0.0.1:8141. On submission, accept a callback_url as well, and on completion POST to that address. In /root/aj/hook.log, the job_id and status=succeeded are recorded.

Notes

• 202 means 'received', not 'done'. Without a Location, the contract is incomplete.
• If you put a TTL on the state of completed jobs and give 410 Gone instead of 404 for expired lookups, the client can tell them apart.
• Common mistake 1: using the ID of a result resource that has not yet been created as the job id — if it fails, a ghost ID remains.
• Common mistake 2: attaching Retry-After to the completed response too, making the client poll forever.

Start the job submission API

Start /root/aj/api.py on 127.0.0.1:8140. POST /jobs receives {"n":5} and puts it in the queue q:jobs2.

Submission puts it in the queue and answers right away. Do not do the actual processing — that is the heart of asynchrony.

Return 202 and the Location header

The POST /jobs response is 202 with the Location: /jobs/<job_id> header and the body {"job_id":"...","status":"queued"}.

The contract holds only when "received" and "where to check" come together. You need both the header and the body.

Build the status lookup endpoint

GET /jobs/<job_id> returns 200 and a status. The value is one of queued, running, succeeded, and failed. A nonexistent id returns 404.

Four values are enough for the status vocabulary. Give 404 for a nonexistent job id.

Have the worker transition the state

/root/aj/worker.py takes an item out of the queue, changes its state to running, and after processing changes it to succeeded.

When taken out of the queue it is running, and when done it is succeeded. It is convenient to keep the state in a Redis hash.

Include the result after completion

The response for looking up a completed job must have a result key. In the queued state there must be no result key.

If the result is small, you may include it in the status response. Before completion there must be no result key.

Suggest a polling interval

A queued or running response carries the Retry-After header as integer seconds, and a succeeded response does not.

Give the header only while in progress. If a completed response has this header, the client keeps polling.

Make submission idempotent

If you submit twice with the same Idempotency-Key, it returns the same job_id and only 1 item goes into the queue.

If the connection drops before receiving 202, the client retries. Give the same identifier for the same key.

Send a completion webhook

Start /opt/app/hooksink.py on 127.0.0.1:8141. On submission, accept a callback_url as well, and on completion POST to that address. In /root/aj/hook.log, the job_id and status=succeeded are recorded.

Start the receiver first and accept the callback address along with the submission. The receipt record must exist to be graded.