Building a Gate That Blocks the Merge

Goal

You build a test runner, a coverage gate, retries and quarantine in shell, and combine these signals into a single merge verdict.

Why it matters

The worth of a gate comes not from the ability to let things through but from the ability to block. So the runner must count failures accurately, the coverage comparison must not wobble at the boundary value, and retries must have an upper bound. Infinite retry hides failures, and quarantine is a compromise that keeps unstable tests from blocking the whole team while leaving that fact in the report. The reason the final report matters is that it is not a person but an automatic mechanism such as the required status checks of branch protection that reads this verdict and blocks the merge. A report whose figures and verdict disagree makes the gate meaningless.

Steps

Create /root/ci2/run-tests.sh <디렉터리> and give it execute permission. It runs the *.sh files in the directory received as the argument one by one, and at the end prints a one-line summary passed=<수> failed=<수> skipped=<수>. If everything passes, the exit code is 0. A test file may have no execute permission, so run it in the form bash "$f".
Even if failing tests are mixed in, run the rest to the end and then finish with a non-zero code at the end. If there are 2 passes and 1 failure, the summary must be passed=2 failed=1.
If given an empty directory, output passed=0 failed=0 and exit code 0. At that time, do not create files such as logs inside the target directory.
Create /root/ci2/gate-coverage.sh <커버리지파일> <임계값>. The coverage file is a single line in the format covered=80 total=100. The percentage is covered * 100 / total, and it gives exit code 0 if it is at or above the threshold and a non-zero code if below (equal passes). On failure, output the actual percentage value.
Create /root/ci2/retry.sh <최대시도횟수> <명령...>. If the command succeeds, it stops immediately and gives 0, and if it fails every time up to the maximum count, it gives a non-zero code. The actual number of attempts never exceeds the maximum count.
If the target directory has a quarantine.txt, the file names written there (one per line) are not run and are counted as skipped. Quarantined tests are not counted as failed, and if everything else passes, the exit code is 0.
Create /root/ci2/gate.sh <테스트디렉터리> <커버리지파일> <임계값>. It gives exit code 0 only when both the test and coverage conditions pass, and gives a non-zero code if even one fails.
Create /root/ci2/gate-report.json. The fields are tests.passed, tests.failed, tests.skipped, coverage.pct, coverage.threshold, coverage.ok (boolean true/false) and decision (the string merge or block). If failed is 1 or more or pct is below threshold, decision must be block, and otherwise merge.

Notes

The summary line is checked after whitespace is removed, so do not put spaces next to the equals sign: passed=2 failed=0 skipped=0.
If you print intermediate progress in the form passed= several times, the check gets confused. Print the tally line only once, at the end.
The test files that grading creates have no execute permission. If you run them directly with "$f", they are all counted as failures.
If you use shell integer arithmetic for the coverage comparison, the decimals are cut off and it is wrong at the boundary. Use awk to compare real numbers.
Common mistakes: the runner dying at the first failure so it cannot count the rest, counting quarantined tests as failed, and writing the decision by hand so that it disagrees with the figures.

Build a test runner

Create /root/ci2/run-tests.sh <디렉터리> and give it execute permission. It runs the *.sh files in the directory received as the argument one by one, and at the end prints a one-line summary passed=<수> failed=<수> skipped=<수>. If everything passes, the exit code is 0. A test file may have no execute permission, so run it in the form bash "$f".

It is called as /root/ci2/run-tests.sh <디렉터리>. Run the *.sh files in the directory one by one and print only one summary line at the end. The test files that grading creates have no execute permission, so you must run them with bash "$f".

Count to the end even on failure

Even if failing tests are mixed in, run the rest to the end and then finish with a non-zero code at the end. If there are 2 passes and 1 failure, the summary must be passed=2 failed=1.

If you stop at the first failure, you do not see the whole picture. Run to the end and then finish with a non-zero code at the end. If you call the test as it is with set -e turned on, the runner itself dies. Handle the exit code directly with if bash "$f"; then ... else ... fi.

Safe even with empty input

If given an empty directory, output passed=0 failed=0 and exit code 0. At that time, do not create files such as logs inside the target directory.

Having no tests at all is not a failure. When the glob finds nothing, check for existence so that the string *.sh itself is not passed on as a file name. You must not create a log file inside the target directory.

A coverage threshold gate

Create /root/ci2/gate-coverage.sh <커버리지파일> <임계값>. The coverage file is a single line in the format covered=80 total=100. The percentage is covered * 100 / total, and it gives exit code 0 if it is at or above the threshold and a non-zero code if below (equal passes). On failure, output the actual percentage value.

It is gate-coverage.sh <파일> <임계값> and the file is a single line covered=80 total=100. The boundary value (when equal) passes. Comparing with integer division is wrong at the boundary, so compare as real numbers like awk 'BEGIN{exit !(p+0 >= t+0)}'.

Retries with an upper bound

Create /root/ci2/retry.sh <최대시도횟수> <명령...>. If the command succeeds, it stops immediately and gives 0, and if it fails every time up to the maximum count, it gives a non-zero code. The actual number of attempts never exceeds the maximum count.

It is retry.sh <최대횟수> <명령...>. If it succeeds, stop immediately, and never exceed the maximum count. Infinite retry is not stabilization but hiding failures, so if it fails to the end, it must give a non-zero code.

Quarantine unstable tests

If the target directory has a quarantine.txt, the file names written there (one per line) are not run and are counted as skipped. Quarantined tests are not counted as failed, and if everything else passes, the exit code is 0.

In the quarantine.txt of the target directory, file names are written one per line. Do not run those files and count them as skipped. Quarantined tests are not put into failed, so if the rest pass, the exit code is 0.

A composite gate

Create /root/ci2/gate.sh <테스트디렉터리> <커버리지파일> <임계값>. It gives exit code 0 only when both the test and coverage conditions pass, and gives a non-zero code if even one fails.

gate.sh <테스트디렉터리> <커버리지파일> <임계값> calls the two scripts you built earlier and only combines the verdicts. If even one fails, it is a failure. If you keep the exit code of each call in a variable, you can also print which side blocked.

A report whose verdict and figures agree

Create /root/ci2/gate-report.json. The fields are tests.passed, tests.failed, tests.skipped, coverage.pct, coverage.threshold, coverage.ok (boolean true/false) and decision (the string merge or block). If failed is 1 or more or pct is below threshold, decision must be block, and otherwise merge.

The decision in /root/ci2/gate-report.json must be derived from the figures. If failed is 1 or more or pct is below threshold, it is block; otherwise merge. Put coverage.ok in as a JSON boolean (true/false), not a string.