TT Lab
Get started
Learn Learning paths Courses

CI/CD Pipelines

It was green on the branch and red right after the merge

Continue in TT Lab

Goal

Using a local bare repository as the server, you enforce a protected branch with receiving-side hooks, set up a required check with per-commit check records, preserve linear history, measure branch age, reproduce a semantic conflict, and then carry green through to after the merge with a merge queue.

Why it matters

A branch's green is the result of looking at 'my change plus the trunk at that time'. If another branch goes in first, that result is no longer a guarantee about the present trunk. So what was green can turn red after the merge, and what breaks then is not one person's branch but the trunk that everyone stands on. There are only two places to block it. One is the receiving side's rules — a rule placed on the pushing side does not come along with a clone and anyone can turn it off, so it cannot be enforcement. The other is a recheck right before merging — lining things up, putting each on top of the tip of the present trunk, checking again, and putting in only what passes. This lab has you build those two yourself with git alone, inside a Pod with neither internet nor a hosting service. A hosting service's branch protection and merge queue are names given to this framework, and only someone who has built it once by hand knows what those features guarantee and what they do not.

Steps

  1. Start at /root/trunk. First create the bare repository /root/trunk/server.git, which will play the role of the 'server', with the default branch main, and create an empty directory /root/trunk/server.git/status in which to write check results per commit. Then clone that server to /root/trunk/app, set the identity to dev / dev@example.com, create three Python files and a test runner, commit them and push straight to main — calc.py (discounted(price, rate) returns price - price * rate), checkout.py (cart_total(prices, rate) adds up the discounted price of each value), tests.py (asserts discounted(100, 0.2) == 80 and cart_total([100, 200], 0.2) == 240 and prints ok), run-tests.sh (moves to its own directory and runs python3 tests.py, needs execute permission). Finally clone the same server once more to /root/trunk/app2, set the identity to other / other@example.com, add README.md, and push also straight to main. Then save the output of git --git-dir=/root/trunk/server.git log --format='%h %ae %s' main as it is to /root/trunk/evidence/01-direct-push.txt.
  2. Create the server hook /root/trunk/server.git/hooks/update and give it execute permission. The arguments, in order, are the reference name to update, the old value and the new value. If the reference is refs/heads/main, it prints one line of guidance on what to do on standard error and ends with 1, and for any other reference it ends with 0 without a word. Next create the local hook /root/trunk/app/.git/hooks/pre-commit and give it execute permission — if the staged changes contain DO-NOT-COMMIT, it rejects, and otherwise it lets it pass. Now confirm three things and write them in /root/trunk/evidence/02-hooks.txt as three lines of <이름> <값> (name, value). main-push is the result when you make one commit in /root/trunk/app and push straight to main (accepted or rejected), same-push-both is what reached the server when you pushed main and a feature branch together with a single push command (one of all, feature-only and none), and cloned-hook is whether, when you freshly cloned the server to /root/trunk/app3, that copy had a pre-commit hook (present or absent).
  3. First create the check runner /root/trunk/ci-check.sh <server.git> <커밋SHA> (server, commit SHA) and give it execute permission. It extracts the tree of that commit from the server into a temporary directory (git --git-dir=<server> archive <커밋>), runs run-tests.sh, and writes the result in the first line of the file <server.git>/status/<커밋 40자리 SHA> (40-character commit SHA) as pass or fail. On the screen it prints one line <40자리 SHA> <pass|fail>, and the exit code is 0 for pass, 1 for fail, and 2 if arguments are missing or the commit is not on the server. It does not leave the temporary directory. Then fix /root/trunk/server.git/hooks/update to apply these rules to refs/heads/main — reject an update whose new value is only zeros (deletion), reject an update whose old value is not an ancestor of the new value (a forced update), and, for each new commit obtained with git rev-list <옛값>..<새값> (old value..new value), reject if there is no record and reject if the first line is not pass. Put the 40-character SHA of the offending commit in the rejection message. Finally, from /root/trunk/app push two feature branches to the server — one a change where the tests pass, one a change where a deliberately failing assertion is added to tests.py — run ci-check.sh on each, and then write two lines in /root/trunk/evidence/03-status.txt as green <40자리 SHA> pass and red <40자리 SHA> fail.
  4. Add one more rule to /root/trunk/server.git/hooks/update — if among the new commits coming into refs/heads/main there is a commit with two or more parents (a merge commit), it prints on standard error a message that contains both that commit's 40-character SHA and the word rebase, and rejects. It must reject even if all the check records are green. Then confirm that the rule actually runs — in /root/trunk/app, create a branch diverging from origin/main, feature/merge-demo, put one commit on it, merge origin/feature/green into it with --no-ff, manually put pass records on all the new commits, and push that branch tip to main. Strip the remote: prefix from the first line of the rejection message and save it as it is to /root/trunk/evidence/04-linear.txt.
  5. Create /root/trunk/branch-age.sh <server.git> [최대일수] (server, optional maximum days) and give it execute permission. The default for the maximum days is 3. It sweeps only the server's refs/heads/feature/* in ascending name order and, for each branch, prints one line <브랜치이름> <앞선커밋수> <나이> <OK|STALE> (branch name, commits ahead, age, OK or STALE). The branch name is with refs/heads/ removed, the commits ahead is the number of commits in main..<브랜치>, and the age is the number of days (rounded down) between the committer time of the oldest commit in that range and now. If the age exceeds the maximum days it is STALE, otherwise OK. If there is even one STALE, the exit code is 1, if none 0, and if arguments are missing or the server does not exist, 2. Then push two more feature branches from /root/trunk/app — feature/old-report, whose first commit has author and committer time 40 days ago, and feature/quick-fix, which you create now. Finally, save the output of /root/trunk/branch-age.sh /root/trunk/server.git 3 to /root/trunk/evidence/05-branch-age.txt.
  6. From /root/trunk/app, create two feature branches that diverge from origin/main and push them to the server. The files the two branches modify must not overlap — only then does it show that they merge cleanly and still break. feature/sem-a modifies only calc.py so that discounted rounds and returns the result with round(...). feature/sem-b adds, in checkout.py, coupon_total(price) (= discounted(price, 0.15)) and, in tests.py, the assertion coupon_total(105) == 89.25. Run /root/trunk/ci-check.sh on each of the two branches and leave green records. Then clone the server into a temporary directory, try rebasing feature/sem-b onto feature/sem-a (there must be no file conflict) and run run-tests.sh on the result. Write what you confirmed in four lines in /root/trunk/evidence/06-semantic.txt — sem-a PASS, sem-b PASS, merged <PASS|FAIL> and textual <clean|conflict>.
  7. Create /root/trunk/merge-queue.sh <server.git> <대기열파일> (server, queue file) and give it execute permission. The queue file has one branch name per line, and blank lines and lines starting with # are skipped. For each item, in order, it does this — it fetches the tip of the present main and rebases the branch on top of it (if it cannot be put on top, REJECTED <브랜치> conflict), first pushes the commits put on top to refs/queue/<브랜치>, runs the ci-check.sh in the same directory on each of those commits (if even one fails, REJECTED <브랜치> tests), and if everything is green, pushes that tip to main (if rejected, REJECTED <브랜치> push) and prints MERGED <브랜치> <40자리 SHA> (branch, 40-character SHA). A branch not on the server is REJECTED <브랜치> missing. A failed item is dropped and it continues to the next item. When it finishes, it does not leave refs/queue/*. The exit code is 0 if everything is merged, 3 if anything was rejected, and 2 if the arguments or queue file are wrong. Then, in /root/trunk/queue.txt, write feature/sem-a, feature/sem-b and feature/quick-fix in this order, run the queue, and save the output to /root/trunk/evidence/07-queue.txt.
  8. Create /root/trunk/protection-report.sh <server.git> <보고서파일> (server, report file) and give it execute permission. Take a fresh copy of the server for each probe so that the original server is not changed by a single character. You actually push five things in this order — force-main (rewrite the tip commit and force-update), no-status (a new commit with no record), failed-status (a new commit with its record left as fail), merge-commit (a merge commit with pass recorded for all new commits), queue-merge (a linear commit with pass recorded for all new commits). For each probe, write one line <탐침이름> <BLOCKED|ALLOWED> <거절 메시지 첫 줄 또는 -> (probe name, BLOCKED or ALLOWED, the first line of the rejection message or a dash), and on the last line write SUMMARY blocked=<수> allowed=<수> (counts). The rejection message is the first line of the push's standard error that starts with remote: , with the prefix removed. Leave the report as a file and also print it on the screen. If the parent directory does not exist, create it. The exit code is 0 if the first four are all blocked and only queue-merge passed, 3 otherwise, and 2 if the arguments are wrong. Finally, run /root/trunk/protection-report.sh /root/trunk/server.git /root/trunk/report/protection.txt and leave the report.

Notes

With no rules, anyone could push straight to the trunk

Start at /root/trunk. First create the bare repository /root/trunk/server.git, which will play the role of the 'server', with the default branch main, and create an empty directory /root/trunk/server.git/status in which to write check results per commit. Then clone that server to /root/trunk/app, set the identity to dev / dev@example.com, create three Python files and a test runner, commit them and push straight to main — calc.py (discounted(price, rate) returns price - price * rate), checkout.py (cart_total(prices, rate) adds up the discounted price of each value), tests.py (asserts discounted(100, 0.2) == 80 and cart_total([100, 200], 0.2) == 240 and prints ok), run-tests.sh (moves to its own directory and runs python3 tests.py, needs execute permission). Finally clone the same server once more to /root/trunk/app2, set the identity to other / other@example.com, add README.md, and push also straight to main. Then save the output of git --git-dir=/root/trunk/server.git log --format='%h %ae %s' main as it is to /root/trunk/evidence/01-direct-push.txt.

A bare repository is a repository with no working tree, so it is good for receiving pushes. With git init --bare -b main you can even set the default branch name. The Pod has no git identity, so you must first set git config user.name and user.email for each repository for commits to work. The facts to confirm in this step are that there are no rules at all right now, and so two people can each directly modify the trunk.

Once the rule was put on the receiving side, direct pushes were blocked

Create the server hook /root/trunk/server.git/hooks/update and give it execute permission. The arguments, in order, are the reference name to update, the old value and the new value. If the reference is refs/heads/main, it prints one line of guidance on what to do on standard error and ends with 1, and for any other reference it ends with 0 without a word. Next create the local hook /root/trunk/app/.git/hooks/pre-commit and give it execute permission — if the staged changes contain DO-NOT-COMMIT, it rejects, and otherwise it lets it pass. Now confirm three things and write them in /root/trunk/evidence/02-hooks.txt as three lines of <이름> <값> (name, value). main-push is the result when you make one commit in /root/trunk/app and push straight to main (accepted or rejected), same-push-both is what reached the server when you pushed main and a feature branch together with a single push command (one of all, feature-only and none), and cloned-hook is whether, when you freshly cloned the server to /root/trunk/app3, that copy had a pre-commit hook (present or absent).

If a hook has no execute permission, git simply skips it without even an error — a state in which nothing is blocked while you believe the rule is on arises here. update runs once for each reference to be updated, and if it is non-zero it blocks only that reference. With pre-receive, the whole receiving operation would be rejected at once — if you try pushing two references together, the difference between the two shows up as it is. A local hook is in $GIT_DIR/hooks and does not propagate by clone.

If you only lock it, no one can get in — accept only green commits

First create the check runner /root/trunk/ci-check.sh <server.git> <커밋SHA> (server, commit SHA) and give it execute permission. It extracts the tree of that commit from the server into a temporary directory (git --git-dir=<server> archive <커밋>), runs run-tests.sh, and writes the result in the first line of the file <server.git>/status/<커밋 40자리 SHA> (40-character commit SHA) as pass or fail. On the screen it prints one line <40자리 SHA> <pass|fail>, and the exit code is 0 for pass, 1 for fail, and 2 if arguments are missing or the commit is not on the server. It does not leave the temporary directory. Then fix /root/trunk/server.git/hooks/update to apply these rules to refs/heads/main — reject an update whose new value is only zeros (deletion), reject an update whose old value is not an ancestor of the new value (a forced update), and, for each new commit obtained with git rev-list <옛값>..<새값> (old value..new value), reject if there is no record and reject if the first line is not pass. Put the 40-character SHA of the offending commit in the rejection message. Finally, from /root/trunk/app push two feature branches to the server — one a change where the tests pass, one a change where a deliberately failing assertion is added to tests.py — run ci-check.sh on each, and then write two lines in /root/trunk/evidence/03-status.txt as green <40자리 SHA> pass and red <40자리 SHA> fail.

When a hook runs inside a bare repository, git rev-parse --git-dir points to that repository — do not pin the record location as an absolute path. The same rule must run in copies too. git merge-base --is-ancestor A B ends with 0 if A is an ancestor of B. Assume the place where records are left is somewhere only CI can write — in a real service, that place is exactly the 'required status check'. The check runner must look at a commit that has come into the server, not a working copy. A commit that is not yet on the server cannot be extracted.

Send merge commits back and make them rebase

Add one more rule to /root/trunk/server.git/hooks/update — if among the new commits coming into refs/heads/main there is a commit with two or more parents (a merge commit), it prints on standard error a message that contains both that commit's 40-character SHA and the word rebase, and rejects. It must reject even if all the check records are green. Then confirm that the rule actually runs — in /root/trunk/app, create a branch diverging from origin/main, feature/merge-demo, put one commit on it, merge origin/feature/green into it with --no-ff, manually put pass records on all the new commits, and push that branch tip to main. Strip the remote: prefix from the first line of the rejection message and save it as it is to /root/trunk/evidence/04-linear.txt.

git rev-list --parents -n 1 <커밋> prints that commit and its parents on one line — if there are three or more words, it is a merge commit. The reason to require linear history is not a matter of taste. Reverting and narrowing down the culprit become noticeably easier, and the question 'did this commit pass on the trunk?' can be answered with a single commit. A rejection is half of blocking — the other half is telling what to do now.

Branches that last several days were creating most of the problems

Create /root/trunk/branch-age.sh <server.git> [최대일수] (server, optional maximum days) and give it execute permission. The default for the maximum days is 3. It sweeps only the server's refs/heads/feature/* in ascending name order and, for each branch, prints one line <브랜치이름> <앞선커밋수> <나이> <OK|STALE> (branch name, commits ahead, age, OK or STALE). The branch name is with refs/heads/ removed, the commits ahead is the number of commits in main..<브랜치>, and the age is the number of days (rounded down) between the committer time of the oldest commit in that range and now. If the age exceeds the maximum days it is STALE, otherwise OK. If there is even one STALE, the exit code is 1, if none 0, and if arguments are missing or the server does not exist, 2. Then push two more feature branches from /root/trunk/app — feature/old-report, whose first commit has author and committer time 40 days ago, and feature/quick-fix, which you create now. Finally, save the output of /root/trunk/branch-age.sh /root/trunk/server.git 3 to /root/trunk/evidence/05-branch-age.txt.

You can set commit times with GIT_AUTHOR_DATE and GIT_COMMITTER_DATE, which accept the @<에포크초> (epoch seconds) format. git log -1 --format=%ct prints the committer time in epoch seconds. If you give git for-each-ref the argument refs/heads/feature, only what is under it comes out. Sorting depends on the locale, so it is safer to attach LC_ALL=C. Why measure age — once you start measuring, it usually turns out that a few branches that last several days create most of the merge accidents.

Two that were each green turned red when combined

From /root/trunk/app, create two feature branches that diverge from origin/main and push them to the server. The files the two branches modify must not overlap — only then does it show that they merge cleanly and still break. feature/sem-a modifies only calc.py so that discounted rounds and returns the result with round(...). feature/sem-b adds, in checkout.py, coupon_total(price) (= discounted(price, 0.15)) and, in tests.py, the assertion coupon_total(105) == 89.25. Run /root/trunk/ci-check.sh on each of the two branches and leave green records. Then clone the server into a temporary directory, try rebasing feature/sem-b onto feature/sem-a (there must be no file conflict) and run run-tests.sh on the result. Write what you confirmed in four lines in /root/trunk/evidence/06-semantic.txt — sem-a PASS, sem-b PASS, merged <PASS|FAIL> and textual <clean|conflict>.

A text conflict is reported by the tool, so it only costs time and is not missed. What is missed is the case where different lines are modified and they merge cleanly but only the result is wrong. Python's round sends a value whose first decimal digit is 5 to the nearest even number — once rounding gets in, a value that ended in .25 is no longer that value. The check done on the branch looked at 'my change plus the trunk at that time', not at 'the trunk after merging'. To avoid disturbing your working copy, do the trial merge in a temporary clone.

The queue puts it on top of the tip of the present trunk and checks again

Create /root/trunk/merge-queue.sh <server.git> <대기열파일> (server, queue file) and give it execute permission. The queue file has one branch name per line, and blank lines and lines starting with # are skipped. For each item, in order, it does this — it fetches the tip of the present main and rebases the branch on top of it (if it cannot be put on top, REJECTED <브랜치> conflict), first pushes the commits put on top to refs/queue/<브랜치>, runs the ci-check.sh in the same directory on each of those commits (if even one fails, REJECTED <브랜치> tests), and if everything is green, pushes that tip to main (if rejected, REJECTED <브랜치> push) and prints MERGED <브랜치> <40자리 SHA> (branch, 40-character SHA). A branch not on the server is REJECTED <브랜치> missing. A failed item is dropped and it continues to the next item. When it finishes, it does not leave refs/queue/*. The exit code is 0 if everything is merged, 3 if anything was rejected, and 2 if the arguments or queue file are wrong. Then, in /root/trunk/queue.txt, write feature/sem-a, feature/sem-b and feature/quick-fix in this order, run the queue, and save the output to /root/trunk/evidence/07-queue.txt.

Why push to refs/queue/ first — the check runner can extract only commits that have come into the server, and the commits newly made by putting on top are not on the server yet. Only main is protected, so you can freely push to other references. A real service's queue also makes temporary branches like this to check. When the earlier item goes in, the base of the next item changes — so the worth of a queue lies in 'checking again'. If you stop the whole line because one earlier item broke, there is no reason to use a queue.

On one page, where and in what words a rule-breaking push is blocked

Create /root/trunk/protection-report.sh <server.git> <보고서파일> (server, report file) and give it execute permission. Take a fresh copy of the server for each probe so that the original server is not changed by a single character. You actually push five things in this order — force-main (rewrite the tip commit and force-update), no-status (a new commit with no record), failed-status (a new commit with its record left as fail), merge-commit (a merge commit with pass recorded for all new commits), queue-merge (a linear commit with pass recorded for all new commits). For each probe, write one line <탐침이름> <BLOCKED|ALLOWED> <거절 메시지 첫 줄 또는 -> (probe name, BLOCKED or ALLOWED, the first line of the rejection message or a dash), and on the last line write SUMMARY blocked=<수> allowed=<수> (counts). The rejection message is the first line of the push's standard error that starts with remote: , with the prefix removed. Leave the report as a file and also print it on the screen. If the parent directory does not exist, create it. The exit code is 0 if the first four are all blocked and only queue-merge passed, 3 otherwise, and 2 if the arguments are wrong. Finally, run /root/trunk/protection-report.sh /root/trunk/server.git /root/trunk/report/protection.txt and leave the report.

The report must be 'the result of poking', not 'something you write in'. The proof is that if you run it against a server with the hooks removed, all five lines flip to passing — the grader actually does that. A bare repository becomes a copy just by copying the directory. Make use of the fact that the hooks and records come along with the copy too. The shortest way to make a forced update is to rewrite the tip commit. Finally, if you match the five lines you ran to the rules built in the earlier steps one by one, you see at a glance which rule blocks which accident.