Inject failures to find resource leaks: design principles

Summary

You observe normal, repeated-shutdown, and failure paths, and the startup and cleanup of the app lifespan.

Why this matters

There was code that closed the connection when the request succeeded, but when an exception occurred the open connection was left behind. A test that checks only success responses did not reveal this leak. A lifecycle test has to focus on when the resource was opened and closed rather than on the return value.

How it works

Create an independent resource object for each test. Check the number of start and stop calls and the open state, and throw an exception on purpose inside the context. Use TestClient with with to actually run the lifespan, and assert the state even after the with block ends.

학생 테스트 → 정상 구현: 실제 시험 모두 통과
           └→ 계약 위반 구현: 해당 동작에서 실패
수집 실패·0개 실행·강제 종료 ≠ 결함 검출

A worksheet for reading the contract and predicting failures

What follows is not an answer key to memorize an implementation but a step-by-step code review. Each change fragment deliberately breaks the contract. Note that the normal case may still pass after the change. Before running it, predict which input, exception, or state you would observe to expose the difference, and after implementing it, compare that prediction with the result.

1. Make the resource state independent — test

Test the following public contract of the provided service.py: new_resource() is a new dictionary {open:False, events:[]}, and separate calls do not share events. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Do not share a mutable list through a global or a default argument. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

"open":True

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

2. Reject a duplicate start — test

Test the following public contract of the provided service.py: start(resource) raises ValueError if it is already open; otherwise it sets open=True and appends 'open' to events. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Reject the behavior of starting twice and losing one resource. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

if False:
        raise

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

3. Make shutdown idempotent — test

Test the following public contract of the provided service.py: stop(resource) sets open=False and appends 'close' to events only when it is open. If it is already closed, it leaves things as they are. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Even when several cleanup paths overlap, a duplicate close event must not appear. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

resource["events"].append("closed")

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

4. Block the use of a closed resource — test

Test the following public contract of the provided service.py: read(resource) raises RuntimeError if it is closed, and returns {ready:True} if it is open. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: The ready state and the existence of the object are different things. The object can exist and still be closed. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

if False:

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

5. Put finally on the exception path — test

Test the following public contract of the provided service.py: scope(resource) is a contextmanager. On entry it calls start, inside the block it yields resource, and on both success and failure of the block it closes with stop. An exception in the block is propagated. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you write the close only after the yield, that line is never reached when an exception occurs. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

pass

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

6. Connect the app lifespan to the resource — test

Test the following public contract of the provided service.py: lifespan_for(resource) returns an asynccontextmanager function lifespan(app). Inside scope(resource), it sets app.state.resource and yields. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: You do not call the lifespan function itself; you pass it to the FastAPI constructor. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

app.state.resource = dict(resource)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

7. Read the ready state through a real request — test

Test the following public contract of the provided service.py: create_app(resource) uses lifespan_for. GET /ready returns the result of read on app.state.resource. The resource must be closed when the context ends. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: You have to use with TestClient for both the lifespan startup and shutdown to run. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

FastAPI()

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

8. Clean up after a failure following the request — test

Test the following public contract of the provided service.py: exercise(resource, fail=False) calls GET /ready inside with TestClient(create_app(resource)). If fail=True, it raises RuntimeError inside, and otherwise it returns the response JSON. In both cases the resource must be closed. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you bind the normal path and the exception path into the same cleanup structure, you reduce the number of shutdown paths you miss. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

if False:

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

What it looks like in the field

The teaching resource dictionary is an observation device that stands in for a real DB connection pool. In production you also have to design for partial initialization failure, concurrency of the connection pool, cancellation handling, and a shutdown time limit. Instead of writing event strings in a report, you check the state of objects that the learner's code changed as it ran. You may read the provided implementation, but grading uses a separate copy. Do not work around a defect by checking the wording of the source or by modifying files; check the execution results of the public interface.

What you will do in the next lab

Eight steps lead to one runnable result. Make the resource state independent — test → Reject a duplicate start — test → Make shutdown idempotent — test → Block the use of a closed resource — test → Put finally on the exception path — test → Connect the app lifespan to the resource — test → Read the ready state through a real request — test → Clean up after a failure following the request — test.

Each step checks actual return values, exceptions, and state changes, not the fact that a function or file exists. After you see the answer, deliberately change a boundary comparison or the cleanup code and check which tests fail. Explain why the earlier tests are kept in the next steps, and write down one operating condition that this lab does not guarantee.