How many people were affected — impact counted by rules

Goal

From the access log of a payment outage, filter out requests that are not users, merge retries, and count people by the rules to find requests, failures, the error rate, and affected users. Turn those rules into a script that also works on logs it has never seen, and at the end fix the scripts of three teams that produce different numbers.

Why it matters

Impact numbers are values that status updates, compensation, regulatory reports, and SLOs take and use as is. Yet even from the same log, the number varies several times over depending on what you exclude and what you merge. Including health checks makes the outage look larger than it is, using the request count as the people count inflates the impact, and dropping the window's time zone counts the wrong hours. The core of this lab is aligning the rules before arguing about "who is right", and leaving the rules as code.

Steps

Unpack the materials with python3 /opt/fixtures/ic/ickit.py open impact and read the counting rules in /root/ic/impact/README.txt. Write the number of health check lines (path exactly /healthz or /readyz) in access.log as a single number in /root/ic/impact/answers/health.txt.
Write the number of lines in the whole log, excluding health checks, bots (bot, crawler, or spider in the User-Agent, case-insensitive), and the internal range (client IP starting with 10.), as a single number in /root/ic/impact/answers/external.txt.
Write the number of external requests whose first attempt falls in the incident window (window.txt, KST), and among them the number of requests whose last attempt has a status of 500 or above, in /root/ic/impact/answers/requests.txt as two lines, requests=<n> and failed=<n> (lines with the same req are one request).
Under the same rules, compute the error rate in the incident window and the usual error rate from the start of the log until the window starts, as percentages (to two decimal places), and write them in /root/ic/impact/answers/rate.txt as two lines, error_rate=<percent> and baseline_rate=<percent>.
Write the number of people with at least one failed request in the window and the number of people who sent requests in the window in /root/ic/impact/answers/users.txt as two lines, affected_users=<n> and active_users=<n>. Count people by user=, and for requests that are not logged in (user=-), count one session= as one person.
Make /root/ic/impact/impact.sh <log> <start-iso> <end-iso> output three lines, requests=<n>, failed=<n>, and affected_users=<n>, by the README rules (execute permission needed; any language, set by the shebang on the first line). The window arguments take the form 2026-11-04T14:10:00+09:00. The grader runs it on this lab's log and also on other logs and windows generated on the spot.
Running incident ic-impact-dispute raises the incident. With the same log (/srv/impact/access.log) and the same window (/srv/impact/window.txt), the three scripts /srv/impact/bin/statuspage, support, and sre give different affected_users. Fix the three scripts by the README rules so that all three produce the same true value (keep how they are called; writing in the number will fail on other logs). In /root/incident/ic-impact-dispute/report.txt, write three lines, statuspage: <code>, support: <code>, and sre: <code>, and evidence: <defective-line> (one line of a script that reveals the defect). Each script's code is one of the following: includes-probes (health checks, bots, and the internal range not excluded), requests-as-users (the number of failed requests as the number of people), retries-double-counted (failed if any retry failed), sampling-gap (log samples missing), sessions-as-users (logged-in people counted separately per session), window-timezone (the time zone of the window times dropped).

Notes

Materials tool: python3 /opt/fixtures/ic/ickit.py open <name> unpacks the materials into /root/ic/<name>/. The materials are generated from a random seed that differs per person, and the grader recomputes the answers from the same seed, so someone else's numbers will not match. Unpacking again gives the same materials; add --fresh to start over with new materials.
Write your answers under /root/ic/<name>/answers/. They disappear when the session ends, so keep a copy elsewhere if you need one.
Incident tools: incident list shows the incidents you can raise and a one-line symptom for each, and incident show <name> shows the ticket again. Raising the same incident again puts it into its original broken state with new random materials (answers you wrote earlier will no longer match).
The report is /root/incident/<name>/report.txt. Write one line <target>: <cause-code> per target, and at the end add evidence: <line> (one line that showed the cause) as is. The evidence must be one line that exists as is in the materials or the observation records (20 characters or more). The grader looks first at the actual state after your fix and uses the report to confirm your judgment.
One log line: <utc-time> <IP> "<method> <path> <protocol>" <status> <bytes> "<User-Agent>" user=<id|-> session=<id|-> req=<id> attempt=<n>. The User-Agent contains spaces, so splitting only on spaces shifts the fields; a regular expression or quote delimiting is safe.
Compare windows using times with a time zone attached. python's datetime.fromisoformat reads +09:00, and reads Z if you replace it with +00:00.
Two common mistakes: counting lines as requests (retry duplicates), and using the number of failed requests as the number of affected users.

Unpack the materials: how many health check lines

Unpack the materials with python3 /opt/fixtures/ic/ickit.py open impact and read the counting rules in /root/ic/impact/README.txt. Write the number of health check lines (path exactly /healthz or /readyz) in access.log as a single number in /root/ic/impact/answers/health.txt.

The request part has the method, path, and protocol inside quotes. Split on spaces, and the path is the fourth field.

Keep only the external users' lines

Write the number of lines in the whole log, excluding health checks, bots (bot, crawler, or spider in the User-Agent, case-insensitive), and the internal range (client IP starting with 10.), as a single number in /root/ic/impact/answers/external.txt.

The User-Agent is the second quoted field: using the quote as the delimiter (awk -F'"') makes it the fourth field. Searching the whole line for bot can match other fields.

Requests and failures in the window: retries as one

Write the number of external requests whose first attempt falls in the incident window (window.txt, KST), and among them the number of requests whose last attempt has a status of 500 or above, in /root/ic/impact/answers/requests.txt as two lines, requests=<n> and failed=<n> (lines with the same req are one request).

The log is UTC and the window is KST. For each request you must hold the first attempt's time and the last attempt's status separately; two python dicts are convenient.

The error rate and the usual error rate

Under the same rules, compute the error rate in the incident window and the usual error rate from the start of the log until the window starts, as percentages (to two decimal places), and write them in /root/ic/impact/answers/rate.txt as two lines, error_rate=<percent> and baseline_rate=<percent>.

Error rate = failed requests ÷ requests × 100. The denominator and numerator must follow the same rules (external requests, retries as one request). It is a percentage (9.33), not a ratio (0.09).

Count the affected people

Write the number of people with at least one failed request in the window and the number of people who sent requests in the window in /root/ic/impact/answers/users.txt as two lines, affected_users=<n> and active_users=<n>. Count people by user=, and for requests that are not logged in (user=-), count one session= as one person.

The number of failed requests and the number of people differ. A logged-in person with several sessions is still one person.

The rules as a script

Make /root/ic/impact/impact.sh <log> <start-iso> <end-iso> output three lines, requests=<n>, failed=<n>, and affected_users=<n>, by the README rules (execute permission needed; any language, set by the shebang on the first line). The window arguments take the form 2026-11-04T14:10:00+09:00. The grader runs it on this lab's log and also on other logs and windows generated on the spot.

Gather the calculations of the previous three steps into one file that takes arguments. Do not drop the time zone marker of the window arguments.

Incident: three teams' impact numbers differ

Running incident ic-impact-dispute raises the incident. With the same log (/srv/impact/access.log) and the same window (/srv/impact/window.txt), the three scripts /srv/impact/bin/statuspage, support, and sre give different affected_users. Fix the three scripts by the README rules so that all three produce the same true value (keep how they are called; writing in the number will fail on other logs). In /root/incident/ic-impact-dispute/report.txt, write three lines, statuspage: <code>, support: <code>, and sre: <code>, and evidence: <defective-line> (one line of a script that reveals the defect). Each script's code is one of the following: includes-probes (health checks, bots, and the internal range not excluded), requests-as-users (the number of failed requests as the number of people), retries-double-counted (failed if any retry failed), sampling-gap (log samples missing), sessions-as-users (logged-in people counted separately per session), window-timezone (the time zone of the window times dropped).

Direction: the three scripts branched from the same template. Line them up against the README's five rules one line at a time, and one mismatched spot turns up per script. Tools: diff (two scripts side by side), print to see intermediate values even without python3 -X, the impact.sh from the previous step. Concretely: look in turn at the filtering conditions (path, User-Agent, IP), the line that picks the last attempt, the data type that collects people (set or list, and what goes in), and the line that reads the window arguments. After fixing, run all three with the same arguments and confirm the numbers converge on one.