Write a script that exits 0 or 1#
The script must pass on a good commit and fail on a bad one, and nothing else. Wait for the result with a bounded timeout rather than checking once right after the click, since a save that takes a moment would otherwise mark a good commit bad. A Playwright script that opens the page, does the failing step and checks the result is enough:
// repro.mjs: exit 0 when the page works, 1 when the bug is present
import { chromium } from 'playwright';
const browser = await chromium.launch();
const page = await browser.newPage();
await page.goto('http://localhost:4173/settings');
await page.getByRole('button', { name: 'Save' }).click();
const ok = await page.getByText('Saved').waitFor({ timeout: 5000 }).then(() => true, () => false);
await browser.close();
process.exit(ok ? 0 : 1);
Keep the script outside the repository, or in a file no commit in the range touches, so every checkout runs the same test.
Build and serve each commit#
Most front ends need a build before the page exists. Wrap the build, the server and the script in one command:
#!/bin/sh
# bisect-step.sh
npm ci --silent || exit 125 # 125 tells bisect to skip this commit
npm run build --silent || exit 125
npx vite preview --port 4173 & server=$!
i=0; until curl -sf http://localhost:4173/ >/dev/null; do
i=$((i+1)); [ $i -gt 30 ] && { kill $server; exit 125; } # server never came up: skip
sleep 1
done
node ../repro.mjs; result=$?
kill $server
exit $result
Exit code 125 tells git bisect run that the commit cannot be tested, so it skips it rather than marking it bad. Use it for commits that do not build or whose server does not start, so an environment failure is never counted as the bug. git-bisect: bisect run
Run it#
git bisect start <bad-commit> <good-commit>
git bisect run ../bisect-step.sh
git bisect reset
We tried it on a small repository with eight commits, where the sixth changed document.title to a misspelled document.titel in a click handler. Bisect checked out three commits, ran the browser script on each, and named change 6 as the first bad commit.
The number of runs grows with the logarithm of the range: about ten runs cover a thousand commits. If each run builds the app, that is the cost to estimate.
When bisect points at the wrong commit#
- The bug depends on data or the backend, which bisect does not change. Pin them with recorded responses so every commit sees the same server.
- The script is flaky. One wrong pass or fail sends bisect down the wrong half. Run the script several times per commit and fail only if it fails consistently, or make the order deterministic first.
- A dependency changed without a commit, such as an unpinned package or a CDN script. Bisect over your history cannot find it; the lockfile and blocked or replaced third-party requests can.
- The browser changed. Pin the browser version in the script; the browser regression guide covers bisecting the browser instead.
Worked example: a merge that was not the culprit#
This example is fictional. A team's checkout stopped applying coupons sometime in the last 140 commits. The team suspects a large refactor merged on Tuesday.
Bisect with a browser script that applies WINTER10 and checks the total runs eight times and names a one-line commit from Monday that renamed a CSS class. The coupon form's submit handler found its button by that class name. Without bisect, the team would have reviewed the refactor first. The browser script then went into the test suite.
Questions#
Can I bisect deployments instead of commits?
If each deploy is reachable at its own URL, such as preview deployments, yes. Point the script at each URL in order, halving the range by hand or with a small loop.
Should the script use a fresh browser profile each time?
Yes. Playwright's launch() starts one by default. A reused profile carries cached files and storage from the previous commit's run.
What if bisect names a merge commit?
Then each side of the merge passed on its own and the combination fails. Compare the merge with each parent: the change that only exists when both are present is the cause.
Return to reproducing what someone else saw for the other layers a replay needs.