TT Lab
Get started
Learn Learning paths Courses

In Front of an Unfamiliar System

Instead of no, ask what you would like to drop

Continue in TT Lab

Goal

You harden the rules that judge in or out of scope into an ordered file and build a judge. You keep a ledger that records, for each assumption, the result if it breaks, how to verify it, and its validity period, and you find the verifications that have expired. To an out-of-scope request you answer not with a refusal but with a trade-off table.

Why it matters

The scope you set in the first week of the engagement starts to leak from the second week. The agreement is in a drawer, and even when you read it, it is ambiguous whether this request falls under that sentence. If you write the rules as an ordered file and run a judge, to "why is this not allowed?" you can just show one rule, and if the rule is wrong, you fix the rule. It becomes a conversation of fixing a file instead of persuading a person. A request that no rule catches is not out of scope but undecided. If you automatically push what was not caught to out, the gaps in the rules are never revealed. The undecided list is the agenda for the next meeting. Assumptions are the same. You build a plan on something you heard in a meeting and three weeks later find out it was wrong, but by then the schedule is already stacked on top of it. If you write for each assumption what collapses if it breaks, how to verify it, its validity period, and its owner, the assumptions whose period has passed come up as a list. The grader does not trust the numbers you write down. It actually runs your judge with different rules and requests each time and checks the verdicts and the deciding rule, and puts boundary dates into the assumption ledger to check the expiry judgment.

Steps

  1. Create and run /root/scope/gen_engagement.py to produce requests.json (25 requests), baseline.json (10 items, a 30-day budget), and sow.md.
  2. Read the agreement and write the rules in order in /root/scope/scope_rules.json. Each rule has id, effect, match, and reason, and you put the narrow rules on top.
  3. Create /root/scope/judge.py so that for each request it sorts into in, out, or undecided by the first rule that matches and writes verdicts.json.
  4. Add action and tag to match so that a rule catches a request only when all the conditions match. The verdict also records which rule decided it.
  5. In /root/scope/assumptions.json, write at least five assumptions. Each has statement, if_broken, verify_how, verified_on, valid_days, and owner.
  6. Create /root/scope/assumptions.py so that it finds the expired assumptions as of --as-of and writes expired.json.
  7. For the newly arrived out-of-scope request RQ-021-NEW, write the trade-off table in /root/scope/tradeoff.json.
  8. Report in four sections in /root/scope/scope_report.md.

Notes

Unpack the kickoff package

Create and run /root/scope/gen_engagement.py to produce /root/scope/requests.json (25 requests), /root/scope/baseline.json (10 tasks, a 30-day budget), and /root/scope/sow.md.

What you hold at kickoff is the request list, the list of agreed tasks, and the agreement. After unpacking, first read the six lines of sow.md — all the rules of the next step come from there.

Turn the agreement into rules

In /root/scope/scope_rules.json, write the rules in order. Each rule has id (unique), effect (in or out), match (at least one of system, action, and tag), and reason (at least ten characters). There must be at least five rules, and when you run them over the 25 requests, at least one each of in, out, and undecided must come out.

Put the narrow rules on top. Work that touches personal data is out on any system, so it goes at the very top, and system-level rules are below it. Do not try to cover every request — what was not caught is undecided, not out, and the undecided list shows the gaps in the rules.

Sort into in, out, and undecided

Create /root/scope/judge.py so that it judges each request by the first rule that matches and writes /root/scope/verdicts.json. If no rule matches, verdict is undecided and rule is null.

Walk the request down the rule list from the top and stop at the first rule that matches. If nothing catches it by the end, it is undecided — do not push it to out. Sort verdicts by request id ascending, and count each of the three in counts.

Catch only when all conditions match

Add action and tag to match so that a rule catches a request only when all the conditions match. A tag matches if that value is in the request's tags. In the verdict, write the id of the deciding rule as it is.

If you treat the conditions as OR, a narrow rule behaves like a broad rule and the priority collapses. Look only at the conditions written in the rule, and let any value pass for a condition that is not written. The grader swaps the order of a narrow rule and a broad rule to check whether the priority is kept.

Build the assumption ledger

In /root/scope/assumptions.json, write at least five assumptions. Each has id, owner, statement (at least fifteen characters), if_broken (at least fifteen characters), verify_how (at least ten characters), verified_on (YYYY-MM-DD), and valid_days (from 1 to 365). As of the reference date 2026-09-17, there must be at least one each of an assumption that has already expired and an assumption that is still valid.

When you write what collapses if it breaks, it becomes clear that some assumptions cause nothing if they are wrong, while for others the whole plan sits on top of them. An assumption with no owner is never checked by anyone even after its period passes, so be sure to write owner. Give each assumption a different verification date and validity period.

Find the assumptions whose verification has expired

Create /root/scope/assumptions.py so that it judges expiry with --ledger --as-of --out, and run it with the reference date 2026-09-17 to produce /root/scope/expired.json. The expiry date is the verification date plus valid_days, and it counts as expired only when the reference date is past the expiry date.

Computing the expiry boundary off by a day is a common mistake. On the expiry date itself it is still valid, and from the next day it is expired. In the result you must also write the expiry date computed for each assumption so that the customer can see until when it is valid.

What would you like to drop

For the newly arrived out-of-scope request RQ-021-NEW, write the trade-off table in /root/scope/tradeoff.json. drop is made of tasks in baseline whose priority is not must, and the sum must be at least added_days while removing any one of them makes it fall short. question is one sentence containing all the ids of the tasks to drop, and ends with a question mark.

If you propose dropping generously much more, the customer feels we are trying to reduce the work. Add the big tasks first, stop when the sum exceeds the cost, and then scan that bundle again to see whether anything in it can be removed. Work that must be done is not subject to trade.

Report the scope and the assumptions

In /root/scope/scope_report.md, write four sections: ## 합의한 범위와 판정 결과 ## 판단이 보류된 요청 ## 전제와 만료된 확인 ## 무엇을 빼겠습니까 (the Korean headings mean "The agreed scope and the verdict results", "The requests whose judgment was deferred", "The assumptions and the expired verifications", and "What would you like to drop"). The counts of the three kinds of verdict, the ids of the undecided requests, the ids of the expired assumptions and what collapses if they break, and the numbers of the trade-off table must all appear.

Do not write the report by hand; generate it from the three files verdicts, expired, and tradeoff. Do not hide the undecided list — if there are five undecided items, it means there are five gaps in the rules, and if you report that as it is, the customer helps fill them in.