Why 15% of Pull Requests Used 59% of Our AI Review Budget

197 pull requests, 444 AI reviews, and why 15% of PRs used 59% of the reviews.*

The loop nobody planned

Every pull request in my repository gets an automatic review from Codex. The agent that wrote the PR reads the findings, fixes them, pushes, and asks for another review. This is a good loop. It catches real bugs: auth checks that a route forgot, student text that could reach a log, a migration that would lock a table.

But a loop with no exit condition is not a process. It is a hope.

One PR in early October went through 25 review rounds and 46 commits before it merged. Two others went through 18 and 16. Each round costs a review, a fix turn for the agent, a CI run, and (the hidden part) one more full re-read of a large agent context. Codex review loops were about 5% of my total agent usage.

The data

I pulled every merged PR from 2 to 16 October: 197 PRs and 444 Codex reviews.

The distribution has a long tail. Most PRs converge in one or two rounds. A small set of PRs never converge on their own, and they take most of the review budget.

The interactive below shows the split between the two groups.

Three patterns repeat in the long threads. I read them to see why these PRs did not converge:

1. 01The fix is narrow. The reviewer flags one instance of a fault. The agent fixes that one line. The next round flags the same fault two files away.
2. 02Each fix adds surface. A new guard, a new branch, a new error message, each one a new thing to review.
3. 03The bar never moves. Round 12 is judged like round 1. Style notes and "consider" comments keep the loop alive long after the real bugs are gone.

The rule

I set a hard cap and a moving bar:

• At most 4 rounds. A round is one review of one pushed head commit.
• Rounds 1 and 2: fix every finding, and fix the same fault everywhere else in the change. This kills pattern 1. I also ask the reviewer for a thorough first pass.
• From round 3: fix defects only. A defect is wrong behaviour, a crash, a broken build or test, data loss, a security or privacy hole, or a doc the code contradicts. Everything else gets a reply with a reason or an issue link.
• After round 4: one last push, no new review request, merge when CI is green.

The cap lives in a hook (codex_wait.py, a script that waits for the review and tells the agent what to do next) and in the agent's review skill. The hook is the part that cannot be argued with.

The twist

The PR that added the cap (PR 536) went through the new process itself.

• Rounds 1 to 3 found 6 high-priority findings. All were fixed.
• Round 4, the last allowed round, found a real bug in the cap itself.

The bug: the hook polls GitHub every 2 minutes for the review. If it saw round 4 on one poll, and the next poll failed (a network error, an HTTP 502), the hook forgot that it had seen round 4. When its wait timed out, it told the agent to ask for another review. That would have started round 5, which is exactly what the cap exists to stop.

The fix is one sticky flag: once any poll sees round 4, the hook remembers it for the rest of the wait, whatever later polls return. I added an end-to-end test that fakes GitHub, answers the findings once, then returns HTTP 502 on every later call, and checks that the hook says "final round" and not "ask again". The hook's suite went to 34 tests. The PR merged after round 4 with no fifth review.

What I learned

• Caps need state that survives failure. A limit that is computed fresh on each poll is only as good as the worst poll.
• Review the review process with the same rigour. The bug was in the code that governs reviews, and the reviewer found it.
• Move the bar, not only the count. A cap alone just cuts a loop short. A cap plus "fix the fault class in rounds 1–2" makes loops shorter before they hit the cap.
• Watch the tail, not the median. My median PR used one review. The cost was in the 15% that used five or more.

Source of the numbers

All PR and review counts come from the GitHub API for merged pull requests in the EdudojoAI/Edudojo-MVP repository, 2–16 October 2026. The round-cap bug and its test come from the PR threads and the test_codex_wait.py suite. No student data is used.

Definitions

• Round: one review of one pushed head commit.
• Reviews per PR: Codex reviews counted per merged PR. The mean is 444 reviews over 197 PRs (2.25).
• Share of reviews: the reviews spent on PRs with more than four rounds, divided by all 444 reviews.

Caveats

• The "about 5% of total agent usage" figure for Codex review loops is not in the source table above. Confirm it against the usage data before publishing, or remove it.
• The "8.7 reviews per PR above the cap, against 1.1 below it" figures are derived from the counts in the data table (261 ÷ 30 and 183 ÷ 167). They are not separate measurements.

References

1. 01GitHub REST API, merged pull requests in EdudojoAI/Edudojo-MVP, 2–16 October 2026.
2. 02Hook source: codex_wait.py, and its test suite test_codex_wait.py (34 tests), PR 536.
3. 03Companion pieces: Article 1 (cost forensics) and Article 3 (portable plugin).