Reproduce time boundaries without sleeping: design principles

Summary

You verify the limit window, the retry time, and per-user isolation with a fake clock.

Why this matters

When sleep was put into a rate limit test, it passed on a developer PC and failed on CI. A slow runner shifted the time boundary, and the test itself took a long time. Time is an input to the program, so you have to make it controllable by the caller and step exactly on the boundary.

How it works

Control the values returned by the clock callback with a list. Separate the request on the left edge from the request just inside it, and verify the rounding up of Retry-After. Read the state directly to check that the record does not grow after a rejection and that one key does not use another key's quota.

학생 테스트 → 정상 구현: 실제 시험 모두 통과
           └→ 계약 위반 구현: 해당 동작에서 실패
수집 실패·0개 실행·강제 종료 ≠ 결함 검출

A worksheet for reading the contract and predicting failures

What follows is not an answer key to memorize an implementation but a step-by-step code review. Each change fragment deliberately breaks the contract. Note that the normal case may still pass after the change. Before running it, predict which input, exception, or state you would observe to expose the difference, and after implementing it, compare that prediction with the result.

1. Validate the settings — test

Test the following public contract of the provided service.py: validate_limit(limit, window) allows only a positive int limit (excluding bool) and a positive finite int/float window, and returns (limit, float(window)). Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: bool is a subtype of int. NaN and infinity also have to be rejected separately. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

not isinstance(limit, int)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

2. Exclude the left edge of the window — test

Test the following public contract of the provided service.py: active(history, now, window) returns, as a new list in the original order, only the timestamps greater than now-window. history is a sorted, non-decreasing list of timestamps. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Check the difference between >=, which keeps a timestamp that has exactly expired, and >. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

stamp >= now - window

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

3. Round the waiting time up — test

Test the following public contract of the provided service.py: retry_after(history, now, window) is the larger integer of 0 and the ceil of (the first timestamp + window - now) of a non-empty history that has already been cleaned up. An empty list gives 0. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you send Retry-After as 0 because 0.2 seconds remain, the client requests again immediately. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

int(history[0] + window - now)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

4. Split the records per key — test

Test the following public contract of the provided service.py: history_for(state, key) returns an empty list for a key that does not exist, and a copy of that record if it does. A lookup alone does not modify state. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you return a shared list, the cleanup in one request can change the record of another request. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

state.get(key, [])

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

5. Record only allowed requests — test

Test the following public contract of the provided service.py: admit(state, key, now, limit, window) validates the settings and then cleans up the expired records of that key. If there is room, it appends now and returns (True,0); if full, it does not append and returns (False,retry_after). It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you append a rejected request, the expiry time is pushed back on every retry. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

len(history) > limit

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

6. Validate the client key — test

Test the following public contract of the provided service.py: client_key(value) returns a string of 1–40 ASCII letters, digits, and hyphens as it is, and everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Limit the range of the input so that an unbounded key size cannot put pressure on the memory of the state. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

<= 80

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

7. Build the rejection response — test

Test the following public contract of the provided service.py: limited_response(wait) is a JSONResponse with status 429, body {error:'rate_limited'}, and a Retry-After header set to wait as a string. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Send the status and the header together so that the client knows when to retry. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

status_code=503

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

8. Finish the request flow with virtual time — test

Test the following public contract of the provided service.py: create_app(clock, limit=2, window=10) checks X-Client-ID at GET /work: an invalid key gives 400 {error:'invalid_client'}, an allowed request gives 200 {ok:True}, and an excess request gives limited_response. The state is kept separate for each app. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Do not actually sleep; pass the current time, held in a list, through the clock function. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

admit(state, "shared", clock(), limit, window)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

What it looks like in the field

This is an example for a single worker held in process memory. It does not guarantee a global limit shared by several Pods or the identity of a malicious client. X-Client-ID is a key for testing, so in production the key should come from an authenticated principal. A persistent clock going backward should be avoided by using a monotonic clock, and the clock in this lab is non-decreasing. You may read the provided implementation, but grading uses a separate copy. Do not work around a defect by checking the wording of the source or by modifying files; check the execution results of the public interface.

What you will do in the next lab

Eight steps lead to one runnable result. Validate the settings — test → Exclude the left edge of the window — test → Round the waiting time up — test → Split the records per key — test → Record only allowed requests — test → Validate the client key — test → Build the rejection response — test → Finish the request flow with virtual time — test.

Each step checks actual return values, exceptions, and state changes, not the fact that a function or file exists. After you see the answer, deliberately change a boundary comparison or the cleanup code and check which tests fail. Explain why the earlier tests are kept in the next steps, and write down one operating condition that this lab does not guarantee.