TT Lab
Get started
Learn Learning paths Courses

Testing Tools in Practice

Make lazy export behavior observable: design principles

Continue in TT Lab

Summary

You verify the number of rows consumed, newline escaping, public fields, and the response format.

Why this matters

Export code tested with a small list looked normal, but it converted a large input entirely to a list and exhausted memory. Having the same result does not mean the contract about how it runs is the same. You have to turn lazy consumption into a state you can see.

How it works

Have the input generator leave a trace in a list each time it is consumed. Check that it was 0 times right after creation, 1 time after the first next, and that nothing more is consumed after the cap. Parse the JSON Lines that contain Korean text and newlines again, and also check the Content-Type of the HTTP response and the removal of internal fields.

학생 테스트 → 정상 구현: 실제 시험 모두 통과
           └→ 계약 위반 구현: 해당 동작에서 실패
수집 실패·0개 실행·강제 종료 ≠ 결함 검출

A worksheet for reading the contract and predicting failures

What follows is not an answer key to memorize an implementation but a step-by-step code review. Each change fragment deliberately breaks the contract. Note that the normal case may still pass after the change. Before running it, predict which input, exception, or state you would observe to expose the difference, and after implementing it, compare that prediction with the result.

1. Validate the row contract — test

Test the following public contract of the provided service.py: validate_row(row) returns row when it is a dict whose id is a positive int (excluding bool) and whose name is a non-empty str. Everything else is ValueError. Extra internal fields are allowed. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Distinguish bool from numbers and treat an empty name as an error. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

not isinstance(row.get("id"), int)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

2. Build only public rows — test

Test the following public contract of the provided service.py: project(row) returns a new dict that has only id and name after validate_row. The internal fields of the original are preserved as they are. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: The export path must apply the same public-field policy as an ordinary API. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

"name":row["name"], "internal_cost":row.get("internal_cost")}

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

3. Encode while preserving line boundaries — test

Test the following public contract of the provided service.py: encode_line(row) is a str that JSON-encodes the result of project with ensure_ascii=False, separators=(',',':'), and sort_keys=True, and appends exactly one trailing newline character at the end. A newline inside name must be a JSON escape. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: If you build JSON by appending strings, the format breaks at quotes and newlines. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

ensure_ascii=True

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

4. Validate the output count cap — test

Test the following public contract of the provided service.py: validate_max(value) returns the value as it is only for an int from 1 to 1000 that is not a bool, and everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Require a cap from the caller so that an infinite input is not read to the end by mistake. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

<= 1001

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

5. Consume only the rows you need — test

Test the following public contract of the provided service.py: take_rows(rows, maximum) is an iterator that lazily returns at most maximum rows, using islice or similar. It validates maximum when called, and each next consumes the input only once. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: The moment you turn it into list(rows), you can no longer handle infinite input or very large input. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

islice(list(rows), validate_max(maximum))

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

6. Serialize rows lazily — test

Test the following public contract of the provided service.py: json_lines(rows, maximum=100) yields encode_line for each row received from take_rows. It does not return a string or a list with everything combined. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Keep object selection and representation conversion each as a lazy stage. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

for row in list(take_rows(rows, maximum)):

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

7. Validate the downloaded lines again — test

Test the following public contract of the provided service.py: decode_lines(text) runs json.loads and then validate_row on each non-empty line of splitlines and returns a list. An empty string gives [], and an empty line in the middle is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Distinguish an empty file from a malformed empty record. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

continue

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

8. Complete the HTTP download — test

Test the following public contract of the provided service.py: create_app(rows) returns json_lines(rows, 100) as an application/x-ndjson StreamingResponse at GET /export. rows is a list that can be iterated again. There must be no internal fields, and the content and order of each row must be preserved. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Check, together with a generator test, that the code does not just state streaming in the Content-Type while gathering everything internally. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

media_type="application/json"

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

What it looks like in the field

TestClient buffers the response, so it does not prove the delay of the first byte over the network or the whole memory cap. Whether evaluation is lazy is checked with a separate counting generator. Once the stream has started, it is hard to change the status to a normal error JSON when you meet a bad row. A real service has to decide which to pick among prior validation, a per-row error format, and an abort policy. You may read the provided implementation, but grading uses a separate copy. Do not work around a defect by checking the wording of the source or by modifying files; check the execution results of the public interface.

What you will do in the next lab

Eight steps lead to one runnable result. Validate the row contract — test → Build only public rows — test → Encode while preserving line boundaries — test → Validate the output count cap — test → Consume only the rows you need — test → Serialize rows lazily — test → Validate the downloaded lines again — test → Complete the HTTP download — test.

Each step checks actual return values, exceptions, and state changes, not the fact that a function or file exists. After you see the answer, deliberately change a boundary comparison or the cleanup code and check which tests fail. Explain why the earlier tests are kept in the next steps, and write down one operating condition that this lab does not guarantee.