Review History as a Checklist: Teaching an Agent What Review Keeps Sending Back

The five largest of 26 kinds of finding, from about 300 review comments, September 7 to October 22, 2026. Counted by hand: about right, not exact.

Why this exists

Every team has reviewers who know things that are not written down anywhere. Which edge cases always get missed. Which permission check the API forgets. Which test still passes with the bug in place. They write the same review comment again and again, and each time a ticket goes back to the developer, it costs a day.

AI agents make this worse before they make it better. An agent writes code fast, and it makes the same mistakes a new developer makes, every time, because it does not remember last week's review. I wanted it to arrive at review with those questions already answered.

1. Count what review actually sends back

I went through seven weeks of reviews in one product team: 207 pull requests, about 300 review comments by people other than the author, and 22 times a ticket was sent back from review. Bots and the author's own replies were left out. Then I sorted every comment by what it was really about. They fell into 26 kinds.

The result was not what I expected. The largest kinds were not exotic. Edge cases nobody tried: empty values, zero, an unsaved record, a failed network request. Accessibility: controls that do not work with a keyboard. Permission checks that are missing, or that differ from what the screen allows. And tests that exist but still pass when the bug is put back.

2. Turn each kind into a question, not a rule

The team's rules were already written down. The agent had read them, and still made these mistakes. A rule says what is right. A question makes the agent look at its own change. So each kind of finding became one or more questions, and the agent answers every question that applies to the change with yes, no or N/A, plus one line of evidence.

before review · questions from past findings

For each changed branch, did I test empty, zero, an unsaved record, and the request failing?Evidence: the test names, one per case.

yes

Next to each refused case, is there a test where a user with the right is allowed?A refuse-only test still passes when the rule blocks everyone.

yes

For each test, did I break the code it guards and see this exact test fail?Evidence: the failure output, quoted.

yes

Does the change do anything for a customer that the ticket does not state in words?If yes: a product owner's answer in the ticket.

Four of the questions, shortened. The answers here are examples.

Some questions came straight from a single painful review. A permission test that only checks the refused user passes even when the rule is forced on everyone, so the question asks for the allowed user too. A rule stricter than the screen is also a bug, so there is a question for a user who can do this in the screen today.

3. A fixed path from ticket to human review

The questions sit inside a larger path. The agent takes a ticket through seven stages, and it may only call the work done when 33 checks each have evidence: the command, an excerpt of its output, the commit. A check without evidence did not run.

PickSetupBuildVerifyReviewPR + CIHand-over

4/7 Verify · checks 14/33 · ETA 45–70 min

The agent shows this line at the start of every message. The ETA is corrected after each stage, by how long the last one really took.

Pick: reviews for colleagues come first, because a waiting review blocks someone else. Then the agent checks whether the problem already has a ticket, and whether a more urgent ticket is open.
Build: a failing test first, so the bug is proven before it is fixed. Then the fix.
Verify: the full suite of about 18,000 tests, the way CI runs it. Then again after merging the main branch.
Review: a second agent with no context from the session, using the strongest model, gets the diff and the review questions. Every finding is fixed or answered, and the review runs again, as many rounds as it takes.
PR + CI and hand-over: CI green on the exact commit, every review thread answered, and a report with the evidence for each check.

4. The person decides

The agent asks before every action other people can see: every comment, every push, every pull request, every change to a ticket's state or assignee. It drafts first and shows the draft. A yes for one ticket does not cover the next one. This is slower than letting it run. It also means nobody on the team finds a comment or a ticket move they did not expect, and that is what makes people willing to use it.

5. Keep the list alive

A checklist that does not change becomes a ritual. So every time review still sends a ticket back, the reason goes into the list, with the pull request it came from. The list was last updated from tickets sent back the week after it was written. Some lessons came from the skill's own use: a push after hand-over now moves the ticket back to in progress, because once a build went red while the ticket still waited for review.

6. What it has done so far

On one day, October 22, six tickets reached colleague review.
A failed PDF upload went from analysis to hand-off in 95 minutes, all tests included.
On another ticket, the second agent reviewed five rounds before a person saw the change.
Four people on the team use it.

What this buys

It does not replace the reviewers. It takes away the part of review that repeats, so reviewers stop writing the same comment for the tenth time and spend their time on what needs a person: design, product decisions, the problem nobody saw before. And what they taught the team stays written down, in a place every agent and every new developer reads before they start.