TT Lab
Get started
Learn Learning paths Courses

Testing Tools in Practice

Isolate global state in configuration tests

Continue in TT Lab

Goal

You verify regressions involving invalid environment values, defaults, snapshots, and exposure of secret fields.

Why it matters

A test that ran earlier changed an environment variable and the tests after it failed. Fixing the test order hid the problem, but the bug remained where even a typo in the settings of the real app turned into a normal default. Input isolation and strict parsing have to be checked separately.

Steps

  1. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_bool(value) returns a bool only for true or false, ignoring case. If there is surrounding whitespace or the value is not a string, it is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Prepare this once at the start. Existing files are not overwritten.

mkdir -p /root/work/test-settings-isolation-lab
test -e /root/work/test-settings-isolation-lab/service.py || cp /opt/fixtures/ten_labs/test-settings-isolation-lab/service.py /root/work/test-settings-isolation-lab/service.py
test -e /root/work/test-settings-isolation-lab/test_service.py || cp /opt/fixtures/ten_labs/test-settings-isolation-lab/test_service.py /root/work/test-settings-isolation-lab/test_service.py
cd /root/work/test-settings-isolation-lab
  1. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_port(value) converts a string that has only ASCII digits to an int and returns it if it is 1–65535. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  2. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_timeout(value) converts a string to a float and returns only finite values greater than 0 and at most 30. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  3. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: required_token(env) returns the stripped value when TOKEN is a string and is not empty after strip. A missing or empty value is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  4. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: load_settings(env) is a dictionary with service=SERVICE of env or 'api' if missing, debug=parse_bool(DEBUG, or 'false' if missing), port=parse_port(PORT, or '8000' if missing), timeout=parse_timeout(TIMEOUT, or '5' if missing), and token=required_token. An empty SERVICE is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  5. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: public_settings(settings) is a new dictionary that has only service and debug. It does not modify the original. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  6. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: snapshot(env) returns the result of load_settings. Even if env is modified after the call, the returned settings do not change. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

  7. In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: create_app(env) reads the snapshot immediately, and if the settings are invalid it makes app creation fail with ValueError. GET /info returns only public_settings. Apps created with different env values do not share settings. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Notes

Parse the boolean explicitly — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_bool(value) returns a bool only for true or false, ignoring case. If there is surrounding whitespace or the value is not a string, it is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Prepare this once at the start. Existing files are not overwritten.

mkdir -p /root/work/test-settings-isolation-lab
test -e /root/work/test-settings-isolation-lab/service.py || cp /opt/fixtures/ten_labs/test-settings-isolation-lab/service.py /root/work/test-settings-isolation-lab/service.py
test -e /root/work/test-settings-isolation-lab/test_service.py || cp /opt/fixtures/ten_labs/test-settings-isolation-lab/test_service.py /root/work/test-settings-isolation-lab/test_service.py
cd /root/work/test-settings-isolation-lab

bool('false') is True. Compare against the two allowed strings directly. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/01-contract.sh.

Check the port range — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_port(value) converts a string that has only ASCII digits to an int and returns it if it is 1–65535. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

A successful integer conversion does not mean the value is in the valid port range. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/02-contract.sh.

Make the time limit a finite value — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: parse_timeout(value) converts a string to a float and returns only finite values greater than 0 and at most 30. Everything else is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

NaN behaves unexpectedly in ordinary comparisons, so check isfinite. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/03-contract.sh.

Reject a missing required secret — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: required_token(env) returns the stripped value when TOKEN is a string and is not empty after strip. A missing or empty value is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Do not replace a missing required secret with a sample default. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/04-contract.sh.

Apply defaults only to missing values — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: load_settings(env) is a dictionary with service=SERVICE of env or 'api' if missing, debug=parse_bool(DEBUG, or 'false' if missing), port=parse_port(PORT, or '8000' if missing), timeout=parse_timeout(TIMEOUT, or '5' if missing), and token=required_token. An empty SERVICE is ValueError. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

The default of get and 'value or default' differ in how they treat an empty string. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/05-contract.sh.

Remove secrets from the public settings — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: public_settings(settings) is a new dictionary that has only service and debug. It does not modify the original. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Rather than masking part of the token value, use a contract in which the field itself is not exposed. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/06-contract.sh.

Separate outside changes from the settings — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: snapshot(env) returns the result of load_settings. Even if env is modified after the call, the returned settings do not change. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Separate the settings at app startup from an input dictionary that may change later. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/07-contract.sh.

Check startup failure and the public response — test

In /root/work/test-settings-isolation-lab/test_service.py, test the following public contract of the provided service.py: create_app(env) reads the snapshot immediately, and if the settings are invalid it makes app creation fail with ValueError. GET /info returns only public_settings. Apps created with different env values do not share settings. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Validate at creation so that a configuration error does not first show up on the first request after the server has started. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-settings-isolation-lab/08-contract.sh.