Why this exists
A companion to Implementer, Reviewer, Tester — that piece is the story of one feature. This one is everything underneath that story: the operating model, why a fixed checklist beats trusting an agent's own sense of "done," the codebase habits that make agent work easier or harder, and what's actually worth measuring.
A closed loop, not a straight line. Only scoping, decisions, and catching wrong assumptions stay human — that part was never replaced.
1. The agent doesn't decide when it's done — a fixed list of gates does
An agent reports success the moment it stops working. That's not the same claim as finished. So "finished" gets written down, in full, before the work starts — and it doesn't change while the work is happening. Each gate can fail. On failure, the work goes back a step; it doesn't continue past it. And the gates stay deliberately boring, on purpose: boring means a person can check one without having to read the code behind it.
The list I actually use runs to dozens of checks across the five phases below. Two rules sit underneath all of them and never bend: never push unless the full suite is green, and never hand a change to a person while it's carrying commits no CI run has ever seen.
Five phases, a fixed checklist inside each one. A failed gate sends the work back a step — it doesn't get to carry on.
2. Four roles, and why the reviewer must not share context
Four roles, not one blurred one. Each sees only what it needs, and findings get reported with evidence attached, not just a conclusion.
The divider is the point: a reviewer who shares context with the implementer only confirms their own assumptions.
| Role | Does | Does not |
|---|---|---|
| Implementer | Writes the change and its tests, one small step at a time. | Does not review its own work. |
| Reviewer | Checks the requirement, the standards, and correctness — with the strongest model available. | Does not re-run the tests. |
| Tester | Runs the suite. Checks that test and assertion counts actually moved, not just the pass/fail line. | Does not write the change. |
| Human | Opens the real screen and looks at it. | Does not skip this step. |
The reviewer must never share context with the implementer. A reviewer who wrote the code, or watched it get written, only confirms their own assumptions back to themselves. A reviewer with zero stake in the implementation, told specifically what to check, finds a different class of bug than the author ever will in their own code — and when it finds something, that finding goes back to the implementer to fix. The reviewer's job stops at finding it.
3. What a green suite can't see
A green suite is not a review. It only proves the tests you wrote agree with the code you wrote — it says nothing about the tests you never thought to write. Every row below got through a fully green suite. None of these are hypothetical — each is a real class of defect the suite itself was structurally unable to catch, which is the actual reason a separate gate exists for it.
Four classes of defect a fully green suite let through — each is the reason a separate, specific gate exists.
| Gate | What it catches |
|---|---|
| Static analysis, not just a syntax check | A syntax checker confirms the file parses. It has no idea whether a method actually exists — an invented method name only fails once its exact line runs, and in an error branch that can be never. |
| A confidence check on names and identifiers | An agent writes a plausible method name, a plausible requirement, and a plausible ticket number with exactly the same confidence it writes correct ones. Only the first of those three can ever fail a test on its own. |
| Trace every change back to an actual requirement | A design choice the agent made on its own, written up as something the ticket demanded, sails through every test — the code is right, the justification is invented, and the build stays green. Only tracing the change back to a real line in the requirement finds it. |
| Break the guard on purpose | A guard isn't covered until you've actually watched its test fail. Delete the guard and re-run; if the test still passes, it was never testing the guard in the first place. |
4. Codebase habits that make agent work harder
None of these are bugs. Every example passed review and CI. An agent doesn't skim — it reads every comment as if it were true, and it copies the style of whatever code sits around it. A habit a human reader would just shrug off becomes something an agent actively repeats.
Four of the five habits below, as a shape: a human habit on the left, what an agent does with it on the right.
| Habit | What the agent does with it |
|---|---|
| One fact written in four places — a commit message plus three docblocks | Copies all four. The next time the fact changes, three of the four quietly go stale. |
| "Why we chose this" explained in a docblock instead of the ticket or an ADR | Reads it as current, settled truth. Inline narrative can't be reviewed, can't be tested, and nobody ever goes back to update it. |
| Two styles in one commit — typed on one method, not the next | Can't tell which one is the actual convention, so it copies whichever one it happened to read most recently. |
| A check bypassed instead of fixed | Treats the bypass as the local convention. The real defect goes untracked, and one bypass quietly becomes the pattern. |
| Writing against an old convention while a migration sits mid-review | The conversion cost stays invisible to whoever wrote it, and grows a little with every file added on the old pattern in the meantime. |
One more rule, separate from the table above: no AI attribution anywhere, in a commit, a PR, or a comment. That one isn't about agent behavior — it's about who a commit history is allowed to credit.
5. Tests that actually find bugs
The problem is rarely too few tests — agents write plenty. The problem is tests that keep passing while the code underneath them is broken. One question filters out most of the weak ones: what can actually break in this change?
| Weak test | Strong test |
|---|---|
| Store a value, reload it, confirm it's the same. | Send 250 and 251 characters at a length limit. |
| Tests the ORM, which every field already shares. | Confirms the value is rejected before any database write happens, with the right error key. |
| Survives almost any bug in the feature. | Fails the moment the validation actually breaks. |
None of this calls for more tests. It calls for one habit before writing any test: name, specifically, what would have to be true for it to fail. If the honest answer is "nothing short of a full outage," it isn't testing the thing that actually matters.
6. Confidence and claims
An agent writes an unproven theory in exactly the same confident style it writes a proven fact. From the prose alone, a reader can't tell them apart.
- Before writing "the root cause is X" — was it actually reproduced, or is there just data that happens to be consistent with it?
- Data that's merely consistent with a theory is not confirmation of that theory.
- Not reproduced yet? The honest word is "hypothesis," not "cause."
- A confident, wrong root cause is worse than an honest "not sure yet" — it sends whoever picks this up next to investigate exactly the wrong thing.
This isn't about doubting everything an agent says. It's one habit: separate "I checked this" from "this is consistent with what I'd expect" before either gets written down as fact. The two look identical in finished prose and are not remotely the same claim.
7. Tooling: engineering gaps vs. decision gaps
Not every gap here is a process problem. Some of it is tooling — and it's worth being specific about which is which, because an engineering gap and a decision sitting above engineering need completely different kinds of pressure to move.
- Some capabilities this depends on are blocked by a decision above engineering, not by anything technical. Chasing that with more engineering effort doesn't help — it needs escalation, not another pull request.
- Some of the tooling that makes this possible is real and works, but the setup is genuinely too hard: a hand-edited config file, a raw token pasted into the right place, no guided install. That's a real adoption barrier for anyone without a fair amount of patience, not a footnote to wave away.
- Keep those two categories explicit whenever this comes up. An engineering blocker gets a ticket. A decision blocker gets named and escalated — not quietly reprioritized into a backlog where it looks like nobody's actually waiting on it.
8. Adoption plan
A real onboarding attempt happened exactly once, and nobody currently owns repeating it. That's a real gap, not a detail to gloss over — so here's the actual plan, not just the admission.
- Start by running the method once, start to finish, with someone else driving and me reviewing — not by writing more documentation first. Documentation nobody has tried following yet just moves the same untested assumptions into a longer document.
- Whoever runs it first gets handed the checklist from the actual start of a ticket, not partway through — every rule in it so far was written from a problem found after the fact, and it's genuinely unknown yet whether it holds up when someone follows it from step one.
- Name an owner for the repeat, not just the first run. A method that only works when I personally drive it isn't a method yet — it's a habit.
9. Measurement
What to measure, and specifically what not to. Coverage gets reported on every single run already — close to nobody, including me on plenty of weeks, actually reads it, which is as much an argument for keeping the measured thing simple and specific as any argument for adding more metrics.
- Coverage on changed lines, compared against the base branch — not a whole-project coverage minimum. A flat project-wide threshold against a large, unevenly-tested codebase either blocks everything or means nothing; changed-line coverage is the only version of that number that says something true about the specific change in front of you.
- Test and assertion counts moving in the direction the change implies — not just a pass/fail line. A suite that goes from 40 to 41 tests but keeps the same assertion count is worth a second look before it's trusted.
- What not to chase: a rising raw test count on its own. More tests that don't individually earn their place by the weak/strong distinction above just make the suite slower without making it more trustworthy.