Contract tests for error-response regressions

Goal

You test known errors, unknown errors, private information, and the tracing header.

Why it matters

After an error message was changed, the retry logic of a mobile app broke. The test checked only that an exception occurred and did not compare the meaning of the status and the public body. A regression in which an internal exception is exposed to the user as it is slipped through the same gap.

Steps

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: DomainError(code, message) is a subclass of Exception and stores the code in .code. str(exception) is the message. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Prepare this once at the start. Existing files are not overwritten.

mkdir -p /root/work/test-error-contract-lab
test -e /root/work/test-error-contract-lab/service.py || cp /opt/fixtures/ten_labs/test-error-contract-lab/service.py /root/work/test-error-contract-lab/service.py
test -e /root/work/test-error-contract-lab/test_service.py || cp /opt/fixtures/ten_labs/test-error-contract-lab/test_service.py /root/work/test-error-contract-lab/test_service.py
cd /root/work/test-error-contract-lab

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: status_for(code) is missing=404, conflict=409, invalid=422, and anything else=500. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: public_message(code) is missing='Resource not found', conflict='State conflict', invalid='Invalid request', and anything else='Internal error'. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: request_id(value) returns the value as it is if it is a string of 1–32 characters made only of ASCII letters, digits, underscores, and hyphens, and 'untracked' otherwise. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: problem(code, rid) is a dictionary that has only type='urn:labhub:problem:'+code, title and detail=public_message(code), status=status_for(code), and request_id=request_id(rid). It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: response_for(code, rid) is a JSONResponse with problem as the body, status_for as the status, application/problem+json as the media_type, and X-Request-ID set to the normalized rid. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: install_handlers(app) registers a DomainError handler. It reads the X-Request-ID header and returns response_for for exc.code. It does not put the message of exc into the response. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.
In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: create_app() installs the handlers and raises DomainError(code, internal message) at GET /fail/{code}. But when code=boom it raises RuntimeError. The RuntimeError handler returns a fixed 500 problem response with code=internal. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Notes

You work in the existing lab-dev environment with no internet and no package installation.
Each step runs within a 45-second grading budget. Do not add real sleeps or network calls.
The submitted tests are run in a separate temporary folder against the correct and defective implementations. Against the correct one, every test that actually runs must pass, and against a defective one, the body of a test must fail. A collection error, zero tests run, everything skipped, and forced termination are not a pass. Use only the basic pytest features and the provided libraries.
FastAPI official documentation · pytest official documentation · Python sqlite3
Limitation: The problem response in this lab is a teaching contract with type, title, status, detail, and request_id. It does not claim general internationalization or full standards conformance. A request id is a string used for tracing, not a means of authentication, and secret values must not be recorded as they are in production logs either. You may read the provided implementation, but grading uses a separate copy. Do not work around a defect by checking the wording of the source or by modifying files; check the execution results of the public interface.

Keep a code on the business exception — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: DomainError(code, message) is a subclass of Exception and stores the code in .code. str(exception) is the message. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Prepare this once at the start. Existing files are not overwritten.

mkdir -p /root/work/test-error-contract-lab
test -e /root/work/test-error-contract-lab/service.py || cp /opt/fixtures/ten_labs/test-error-contract-lab/service.py /root/work/test-error-contract-lab/service.py
test -e /root/work/test-error-contract-lab/test_service.py || cp /opt/fixtures/ten_labs/test-error-contract-lab/test_service.py /root/work/test-error-contract-lab/test_service.py
cd /root/work/test-error-contract-lab

Separate the code that a machine decides on from the internal message that a person sees. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/01-contract.sh.

Map the code to a status — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: status_for(code) is missing=404, conflict=409, invalid=422, and anything else=500. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Never treat an unknown code as a success. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/02-contract.sh.

Fix the public sentence — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: public_message(code) is missing='Resource not found', conflict='State conflict', invalid='Invalid request', and anything else='Internal error'. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Even if the exception string contains a DB address or an internal path, it is not exposed. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/03-contract.sh.

Restrict the request id — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: request_id(value) returns the value as it is if it is a string of 1–32 characters made only of ASCII letters, digits, underscores, and hyphens, and 'untracked' otherwise. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Restrict both the length and the character set so that an arbitrary header is not reflected back. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/04-contract.sh.

Build the problem body — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: problem(code, rid) is a dictionary that has only type='urn:labhub:problem:'+code, title and detail=public_message(code), status=status_for(code), and request_id=request_id(rid). It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

If the body and the HTTP status disagree, the client cannot decide which value to trust. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/05-contract.sh.

Make the response format consistent — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: response_for(code, rid) is a JSONResponse with problem as the body, status_for as the status, application/problem+json as the media_type, and X-Request-ID set to the normalized rid. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Returning a plain dictionary can turn even an error into a 200. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/06-contract.sh.

Connect the business exception handler — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: install_handlers(app) registers a DomainError handler. It reads the X-Request-ID header and returns response_for for exc.code. It does not put the message of exc into the response. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

Do not scatter the places where exceptions are caught; put them at the common boundary of the app. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/07-contract.sh.

Hide unexpected errors too — test

In /root/work/test-error-contract-lab/test_service.py, test the following public contract of the provided service.py: create_app() installs the handlers and raises DomainError(code, internal message) at GET /fail/{code}. But when code=boom it raises RuntimeError. The RuntimeError handler returns a fixed 500 problem response with code=internal. It must pass against the correct implementation and be caught, through a failure in the body of an actual test, in an implementation that breaks this contract. Keep the tests from the earlier steps and add a test_ function.

In the test, turn off re-raising of exceptions and check the bytes of the actual 500 response. Do not modify the implementation file. Use pytest.raises to check the expected exception, and assert a concrete expected value for the normal result.

After saving, check with bash /opt/lab/checks/test-error-contract-lab/08-contract.sh.