Isolate global state in configuration tests: design principles

Summary

You verify regressions involving invalid environment values, defaults, snapshots, and exposure of secret fields.

Why this matters

A test that ran earlier changed an environment variable and the tests after it failed. Fixing the test order hid the problem, but the bug remained where even a typo in the settings of the real app turned into a normal default. Input isolation and strict parsing have to be checked separately.

How it works

Pass a fresh environment dict to each test. Separate the case where the environment variable is absent from the case where it exists but is wrong, and check bool strings, port ranges, and a finite timeout. After creating the app, change the original input to check that the snapshot is independent, and assert the key set of the public response.

학생 테스트 → 정상 구현: 실제 시험 모두 통과
           └→ 계약 위반 구현: 해당 동작에서 실패
수집 실패·0개 실행·강제 종료 ≠ 결함 검출

A worksheet for reading the contract and predicting failures

What follows is not an answer key to memorize an implementation but a step-by-step code review. Each change fragment deliberately breaks the contract. Note that the normal case may still pass after the change. Before running it, predict which input, exception, or state you would observe to expose the difference, and after implementing it, compare that prediction with the result.

1. Parse the boolean explicitly — test

Test the following public contract of the provided service.py: parse_bool(value) returns a bool only for true or false, ignoring case. If there is surrounding whitespace or the value is not a string, it is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: bool('false') is True. Compare against the two allowed strings directly. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

bool(value)

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

2. Check the port range — test

Test the following public contract of the provided service.py: parse_port(value) converts a string that has only ASCII digits to an int and returns it if it is 1–65535. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: A successful integer conversion does not mean the value is in the valid port range. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

<= 65536

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

3. Make the time limit a finite value — test

Test the following public contract of the provided service.py: parse_timeout(value) converts a string to a float and returns only finite values greater than 0 and at most 30. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: NaN behaves unexpectedly in ordinary comparisons, so check isfinite. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

0 <= number <= 30

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

4. Reject a missing required secret — test

Test the following public contract of the provided service.py: required_token(env) returns the stripped value when TOKEN is a string and is not empty after strip. A missing or empty value is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Do not replace a missing required secret with a sample default. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

return value

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

5. Apply defaults only to missing values — test

Test the following public contract of the provided service.py: load_settings(env) is a dictionary with service=SERVICE of env or 'api' if missing, debug=parse_bool(DEBUG, or 'false' if missing), port=parse_port(PORT, or '8000' if missing), timeout=parse_timeout(TIMEOUT, or '5' if missing), and token=required_token. An empty SERVICE is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: The default of get and 'value or default' differ in how they treat an empty string. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

env.get("DEBUG","true")

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

6. Remove secrets from the public settings — test

Test the following public contract of the provided service.py: public_settings(settings) is a new dictionary that has only service and debug. It does not modify the original. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Rather than masking part of the token value, use a contract in which the field itself is not exposed. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

"debug":settings["debug"],"token":settings["token"]}

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

7. Separate outside changes from the settings — test

Test the following public contract of the provided service.py: snapshot(env) returns the result of load_settings. Even if env is modified after the call, the returned settings do not change. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Separate the settings at app startup from an input dictionary that may change later. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

return env

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

8. Check startup failure and the public response — test

Test the following public contract of the provided service.py: create_app(env) reads the snapshot immediately, and if the settings are invalid it makes app creation fail with ValueError. GET /info returns only public_settings. Apps created with different env values do not share settings. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Basis for the judgment: Validate at creation so that a configuration error does not first show up on the first request after the server has started. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

Faulty change fragment to review:

return settings

Compare it with the public contract of the function this fragment sits in. If a single success case does not tell them apart, choose as your observation target an input that should be rejected or the state after a failure.

What it looks like in the field

The environment is injected as an ordinary dictionary, so it does not depend on the real process-wide environment. This is not an example that implements an encrypted secret store, key rotation, or dynamic reloading. The token string is teaching input, and you never put a real production key into a lab Pod. You may read the provided implementation, but grading uses a separate copy. Do not work around a defect by checking the wording of the source or by modifying files; check the execution results of the public interface.

What you will do in the next lab

Eight steps lead to one runnable result. Parse the boolean explicitly — test → Check the port range — test → Make the time limit a finite value — test → Reject a missing required secret — test → Apply defaults only to missing values — test → Remove secrets from the public settings — test → Separate outside changes from the settings — test → Check startup failure and the public response — test.

Each step checks actual return values, exceptions, and state changes, not the fact that a function or file exists. After you see the answer, deliberately change a boundary comparison or the cleanup code and check which tests fail. Explain why the earlier tests are kept in the next steps, and write down one operating condition that this lab does not guarantee.