Handling Backpressure and Throughput

Goal

Learn how to translate queue depth into latency, make the system protect itself with a cap and load shedding, and then actually raise the processing rate by adjusting concurrency.

Why it matters

Most incidents in systems with a queue are "the queue was piling up and nobody knew". The API dashboard is all green, the response times are normal, and only the users cannot see their results. If you cannot translate queue depth into delay, you cannot judge how serious this situation is. One line, Little's Law W = L / λ, is enough — with a depth of 3,000 and a processing rate of 50 per second, a user submitting now waits 60 seconds. And once you see on a graph that the wait time grows explosively, not linearly, as utilization approaches 100%, it stays with you why you must not size worker capacity at exactly 100% of the inflow.

Steps

With /opt/app/loadgen.py, put 200 items per second into q:work for 10 seconds. LLEN q:work must be at least 1500.
With /root/qb/measure.sh, measure the queue depth 10 times at 1-second intervals and save it to /root/qb/depth.csv as 11 lines, with the header t,depth.
In /root/qb/little.txt, write the three values L=<깊이> lambda=<초당 처리 건수> W=<초> (L is the depth, lambda is the items processed per second, and W is in seconds). W is L divided by lambda and must be correct to one decimal place.
Start /root/qb/api.py on 127.0.0.1:8142. POST /enqueue returns 429 if the queue length is 1000 or more. Below that, it returns 202.
Put the Retry-After header in the 429 response as integer seconds. In /root/qb/shed.out, write status=429 retry_after=<정수> (an integer).
Process the same 500 items with 1 worker and with 4 workers, and in /root/qb/concurrency.txt write two lines, workers=1 seconds=<값> and workers=4 seconds=<값> (the value is the elapsed seconds). The 4-worker side must be faster.
In /root/qb/report.md, include the three headings ## 안정 조건, ## 목표 이용률, and ## 상한 정책 (stability condition, target utilization, and cap policy), and write at least 25 characters in each section.

Notes

Stability condition: λ < μ. Aim for a utilization ρ = λ/μ of 0.7–0.8.
Little's Law: L = λ x W, inverted W = L / λ
For queue depth alerts, a trend such as "increasing continuously for 5 minutes" is more reliable than an absolute value.
Common mistake: adding only workers without checking that the DB behind them became the new bottleneck.

Fill the queue by applying load

With /opt/app/loadgen.py, put 200 items per second into q:work for 10 seconds. LLEN q:work must be at least 1500.

/opt/app/loadgen.py takes as arguments how many items per second to put in and for how many seconds. You must put in faster than processing for the depth to grow.

Record the queue depth as a time series

With /root/qb/measure.sh, measure the queue depth 10 times at 1-second intervals and save it to /root/qb/depth.csv as 11 lines, with the header t,depth.

Measure the length every second and save it as a CSV. Do not forget the header row.

Estimate the wait time with Little's Law

In /root/qb/little.txt, write the three values L=<깊이> lambda=<초당 처리 건수> W=<초> (L is the depth, lambda is the items processed per second, and W is in seconds). W is L divided by lambda and must be correct to one decimal place.

The time spent in the system is the depth divided by the processing rate. You must write each of the three values to be graded.

Add a queue cap and 429

Start /root/qb/api.py on 127.0.0.1:8142. POST /enqueue returns 429 if the queue length is 1000 or more. Below that, it returns 202.

It is more honest to say you cannot accept it now than to accept and fail to do it. Reject when the cap is exceeded.

Tell the client when to retry with Retry-After

Put the Retry-After header in the 429 response as integer seconds. In /root/qb/shed.out, write status=429 retry_after=<정수> (an integer).

Tell the client in a number when it can come back. Calculating it from the queue depth is more accurate.

Raise worker concurrency to improve the processing rate

Process the same 500 items with 1 worker and with 4 workers, and in /root/qb/concurrency.txt write two lines, workers=1 seconds=<값> and workers=4 seconds=<값> (the value is the elapsed seconds). The 4-worker side must be faster.

Process the same load with 1 worker and with 4 workers and compare the elapsed time. Save both values.

Write up the stability condition

In /root/qb/report.md, include the three headings ## 안정 조건, ## 목표 이용률, and ## 상한 정책 (stability condition, target utilization, and cap policy), and write at least 25 characters in each section.

Write up in a document the relationship between inflow rate and processing rate, the target utilization, and the cap policy. Use the specified headings exactly.