I searched for a whale, but got a cat
Tests are green, so why is the screen awkward?
In one line
Type checking, React DOM behavior checks, and real-browser checks answer different questions. When you report a check result, also explain what was observed.
Why this was needed
The whale search held all the way through, and all the state checks passed. But on a phone the search button is off-screen, and if a response arrives while typing with the keyboard, focus jumps to the results area. The implementer says the tests are green, and the user says it is still unusable. It isn't that one of them is lying; they are looking at different things.
This course separates the scope of checks. The student source goes through type checking, then is rendered with real React in a DOM environment, and the state before and after a request is read. A DOM environment implements some browser APIs, but it is not a layout engine that reproduces text placement or the actual cost of painting. If you judge screen width or performance from that result alone, it becomes unfounded confidence.
How it works
Let's split the behavior to check into small events. Render the component, submit a search, complete a specified response, and read the DOM after the React update has been applied. React's asynchronous act is the tool you use to handle that boundary. Sleeping an arbitrary 3 seconds only wastes time on a fast computer, and on a slow environment you can still read too early.
| Check layer | Example of what to verify | What this alone can't tell you |
|---|---|---|
| TypeScript | Required data and types per state | The actual response arrival order |
| React+DOM | Result after reverse-order responses, effect cleanup | Pixel layout and paint cost |
| Real browser | Keyboard path, narrow screen, render record | Usability with all assistive technologies |
| User observation | Whether they understand the instructions and finish the task | The experience of every other user |
In accessibility, the name of the input, real form submission, announcement of result changes, and focus retention are connected. Announcing new results is different from forcibly moving focus to the results. If the user is still editing the search term, the change must be conveyed while preserving the current input position. Don't present passing an automated check as certification of screen reader usability.
Performance is also viewed by splitting time. The wait from sending a request to receiving the response differs from the cost of computing the results in React and drawing them on screen. If only the network wait is long, adding memo will not make the response come faster. If the results have already arrived and the cost of displaying 10,000 rows at once is large, retrying the network does not solve it.
Don't treat the React Profiler's render time and the browser's layout, paint, and network records as the same number. Before measuring, record the number of data items, development/production build, device, and number of repetitions. Several observations under the same conditions are more useful than one good number. At this stage, don't make up observed values to hit a target number.
How to keep an observation record
A good record tells you the reproduction conditions before anything else, more than a table that lists only result numbers. Fill in the following form with only what you actually did. For items you haven't run yet, don't fill the blank with a plausible value; leave it as unconfirmed.
| Item | What to record |
|---|---|
| Target | Source version, the function you modified, build type |
| Environment | Browser version, screen width, device conditions |
| Input | Number of data items, search term, submission and response order |
| Expectation | The result that should remain at the end and the focus position |
| Observation | Actual result, errors, render or network record |
| Remaining scope | Other screens, assistive technologies, connection to the real service |
For example, a record that says only "I delivered cat after whale" is not enough. It needs the order in which requests were sent, whether it was the mode that ignores cancellation, and whether the past response was delivered after confirming the latest result. Without these conditions, a colleague who receives the same code can't reproduce the problem.
Before and after a performance improvement, too, changing only one condition at a time is easier to interpret. If you change the number of displayed rows, the network delay, and the build type all at once, it is hard to tell which change had the effect. If you reduced the display count, you must tell the user the total number of results and the number currently shown so that they don't mistakenly think data was lost. Quietly discarding results to look fast is a product change, different from a performance improvement.
What it looks like in the field
The same principle applies when you explain the result of an outage fix to a customer. If you say "we reproduced and confirmed reverse-order success and failure and cancellation on unmount, and have not yet confirmed a real mobile session" instead of "tests pass," the next thing to review becomes clear. Showing the scope you haven't confirmed raises trust in the check.
The grading tool is also something to verify. Try feeding it code that returns success unconditionally apart from the correct answer, code that exits midway, and code that never finishes. If you trust only exit code 0 or a PASS line on stdout, student code can pass even if it happens to skip the checks. Also confirm that the grading result was collected to the end and that the student's original was not changed.
What to check yourself
Record the DOM check and the real-browser observation separately. Summarize in a table which check caught which defect, and leave the scope you haven't observed yet. Measure long lists in a real browser, and don't submit times obtained from the DOM model in their place.
For the scope of the tools, see React act, React Profiler, and the jsdom guide.