Write a Change Request and Verify It with a Rehearsal

Goal

Classify the types of three change requests, follow the impact scope of a DB restart to the end of its dependencies, subtract the rollback time backward from the work window to get the stop decision time, then write a JSON change request with those numbers and use a rehearsal record to judge whether the plan was optimistic.

Why it matters

A large share of production outages comes from changes. A change request is a tool for confirming that risk on paper before the work, and the reasons reviewers reject it are usually the same — the rollback criterion is only "if a problem occurs," the impact list has only direct dependencies, and because the rollback time was not calculated, the window looks ample. This lab fills in those three with numbers and builds the habit of verifying the plan with a rehearsal record.

The material is in /opt/lab/capacity/change/, and the grader computes the expected values from the original material.

Steps

1. Copy the five files of /opt/lab/capacity/change/ to /root/chg/.
2. Write the types of requests A, B, and C to /root/chg/01-type.txt as A=, B=, and C= (standard, normal, emergency).
3. Follow the services affected by the restart of db01 to the end of the dependencies and write them to /root/chg/02-impact.txt as impacted=.
4. Write total_min=, rollback_min=, nogo_at=, and fits= to /root/chg/03-window.txt.
5. Write id, type, summary, window, impact, and precheck to /root/chg/cr.json.
6. Add steps (S1…S5, each with verify) and rollback (trigger, max_min, steps) to cr.json.
7. From the rehearsal record, write slowest_step=, over_min=, finished_at=, and past_nogo= to /root/chg/04-rehearsal.txt.

Notes

• Following dependencies: repeat until there is nothing more to add to the impact set.
• Time calculation: datetime.strptime(s, '%H:%M'), timedelta(minutes=n).
• JSON check: python3 -m json.tool /root/chg/cr.json.
• Common mistake 1: judging "work 60 minutes, window 120 minutes, so it is ample" without subtracting the rollback time.
• Common mistake 2: writing rollback.trigger in unmeasurable words such as "restore if anything is wrong."

Copy the material

Copy the five files of /opt/lab/capacity/change/ (requests.txt, inventory.json, plan.csv, window.env, rehearsal.log) to /root/chg/.

Move the directory contents with cp. Start by reading requests.txt.

Classify the change types

Classify each of the three requests A, B, and C in requests.txt as one of standard, normal, and emergency, and write them to /root/chg/01-type.txt as the three lines A=, B=, and C=.

Look at whether it repeats an approved runbook as is, whether it must be newly assessed, and whether it will soon become an outage if not done now.

Follow the impact scope

Follow the depends_on in inventory.json to the end to find every service affected by the restart of db01, and write them to /root/chg/02-impact.txt as one line impacted=<이름들, 쉼표로> (the names, separated by commas; leave out db01 itself).

First find the services that use db01 directly, and then repeat finding the services that use those services until the set stops growing.

Work window and stop decision time

Using plan.csv and window.env, write four lines to /root/chg/03-window.txt: total_min=<minutes 합> (the sum of minutes), rollback_min=<rollback_minutes 합> (the sum of rollback_minutes), nogo_at=<WINDOW_END − rollback_min − BUFFER_MIN, HH:MM> (WINDOW_END minus rollback_min minus BUFFER_MIN, in HH:MM), and fits=<WINDOW_START + total_min 이 nogo_at 이하면 yes, 아니면 no> (yes if WINDOW_START plus total_min is at or before nogo_at, otherwise no).

Take the rollback as the worst case (every step rolled back). Subtracting backward from the end of the window gives the stop decision time.

The head of the change request

Write the change request for request B to /root/chg/cr.json. The top-level object must have id, type (B's type), summary, window ({"start": "02:00", "end": "04:00"}), impact (the list of service names from step 3), and precheck (two or more non-empty sentences, one of which is a backup or snapshot check).

You can check the JSON syntax with python3 -m json.tool cr.json. Do not copy the impact list by hand; read it from 02-impact.txt and put it in.

Verification per step and the rollback criterion

Add steps and rollback to cr.json. steps is a list of objects, in the order of plan.csv, each with an id (S1…S5) and a non-empty verify (the verification method by which you judge it done). rollback is an object with trigger (a measurable criterion that includes the nogo_at time from step 4), max_min (the rollback_min from step 4), and steps (a list of three or more steps).

The trigger must contain a time, rate, or count, not "if a problem occurs." Read the stop decision time from 03-window.txt and put it in.

Read the rehearsal record

Using rehearsal.log, write four lines to /root/chg/04-rehearsal.txt: slowest_step=<계획보다 가장 많이 늦은 단계 id> (the id of the step that ran most behind plan), over_min=<그 단계가 늦은 분, 소수 첫째 자리> (how many minutes that step was late, to one decimal place), finished_at=<마지막 done 의 HH:MM> (the HH:MM of the last done), and past_nogo=<리허설이 끝난 시각이 nogo_at 보다 늦으면 yes, 아니면 no> (yes if the rehearsal ended later than nogo_at, otherwise no).

For each step, subtract the start time from the done time to get the actual duration and compare with the minutes in plan.csv. There are seconds too, so divide by 60 when converting to minutes.