Finding the Culprit Among 1,000 Commits in Ten Steps

One-line summary

The number of checks needed to pin down a single commit in "it worked two months ago but not now" is not the number of commits but log₂(number of commits). For 1,000 commits, it is 10.

Why this is needed

A bug report has come in. "It used to work."

When you open git log, there are 800 commits in between. Checking them one by one takes 800 tries, and skimming by eye biases you toward commits that "look related". And in experience, the culprit is usually a commit that looks unrelated.

git bisect does the binary search automatically.

git bisect start
git bisect bad                 # 지금은 깨져 있다
git bisect good v1.4.0         # 이 버전에서는 됐다
# → git 이 중간 커밋으로 체크아웃한다
# 확인 후
git bisect good   또는   git bisect bad
# 반복. 10번이면 끝난다.
git bisect reset

Automation — bisect run

If you can write the check as a script, nobody even has to sit there.

git bisect start HEAD v1.4.0
git bisect run ./check.sh

check.sh speaks through its exit code.

Exit code	Meaning
0	good
1–124, 126, 127	bad
125	skip — this commit cannot be judged (build failure, etc.)

125 matters. If a commit whose build is broken is mixed in and you judge it bad, the search converges on the wrong place. If you cannot judge, you must return skip.

#!/bin/bash
make build || exit 125          # 빌드 실패 = 판정 불가
./run-test.sh || exit 1         # 테스트 실패 = bad
exit 0                          # 통과 = good

Prerequisites for bisect

For bisection to work well, every commit must be in a state that builds and runs. If there are lots of commits like "WIP", "work in progress" and "commit for now", only the skips increase. The habit of stacking small, complete commits in ordinary times pays off here.

log -S — when did that code come in

It finds only the commits in which a particular string was added or deleted.

git log -S 'timeout=3' --oneline

It answers "when did this setting value change to 3?" in one go. To search with a regular expression, use -G.

You can see the change history of a single file with git log -p -- path/to/file, and if you add --follow, it follows the history even through renames.

blame — not who but why

git blame shows the commit that last modified each line. The purpose is not finding the person responsible but finding context. Why that line was written that way lies in the commit message and the other changes of that commit.

git blame -L 40,60 src/app.py        # 40~60행만
git blame -w                         # 공백 변경 무시
git blame -C                         # 다른 파일에서 옮겨온 코드도 추적

-w is especially useful. In a repository where a formatter has been run once, every line shows up as the "formatting commit", and -w skips that and finds the real change.

reflog — getting back what was lost

git reflog is the record of everything HEAD has moved through. Even if you botched a rebase or deleted a branch, the commit itself stays alive for a while.

git reflog
# a1b2c3d HEAD@{5}: rebase (finish): returning to refs/heads/main
# 9f8e7d6 HEAD@{6}: commit: 잃어버린 작업
git checkout -b rescue 9f8e7d6

The reflog is the basis for the saying "what you commit in git hardly ever disappears".

What it looks like in the field

• You do not know since when performance got worse → bisect run with a benchmark script.
• A strange constant is baked into the code → find the commit that introduced it, and the reason, with log -S.
• A commit disappeared during a rebase → recover with the reflog.

How to break through when bisection gets stuck

git bisect is powerful, but when you actually run it, it gets stuck on a commit that cannot be judged. There are several tricks for that.

Skip a commit that does not build. If you mark it bad, the result goes wrong.

git bisect skip                    # 이 커밋으로는 판단할 수 없다
git bisect skip v2.1..v2.2         # 구간 통째로

If you skip too many, the range does not narrow, so in that case make the judging script return 125 on a build failure. bisect run reads 125 as "skip".

cat > /tmp/check.sh <<'EOF'
#!/bin/bash
make -s build || exit 125         # 판정 불가
./run-test || exit 1              # 나쁨
exit 0                            # 좋음
EOF
git bisect run bash /tmp/check.sh

If there are many merge commits, look only at the first parent. If you skim even the intermediate commits inside feature branches, it takes long, and those commits were never tested in the first place.

git bisect start --first-parent BAD GOOD

If the symptom is intermittent, bisection lies. If it is a problem that occurs once in ten times, make the judging script try several times. If it is still probabilistic, it is faster to find when that code came in with log -S than with bisection.

git log -S 'setTimeout(retry' --oneline -- src/
git log -L :handleRetry:src/client.js      # 그 함수의 변천사만

Always undo it when you finish. If you do not run git bisect reset, you remain on a detached HEAD, and commits made in that state are hard to find later.

Undoing is not the goal. If you just revert the culprit commit as soon as you find it, the other problem that commit was fixing comes back to life. The value of bisection lies in going on to look at why that change produces this symptom and then fixing it.

What you will do in the next lab

You will build a 40-commit history yourself and find the culprit. In that history one commit that cannot be judged is mixed in, so if you do not use 125, bisect confidently points at the wrong commit. You will run the two scripts side by side and see the difference with your own eyes.

And when you use 125 properly, this time bisect narrows only down to two candidates and stops. The last step is decided by a person — where automation ends is the conclusion of this lab.