Why review is now the bottleneck
AI coding tools made writing code cheaper. They did not make reading it cheaper. Teams that adopted assistants and agents often find that pull requests got larger and more frequent, while the number of people who can judge them stayed the same. Senior engineers spend their days reviewing, and some of what they approve contains defects that a smaller change would have made obvious.
Two tempting answers make it worse. One is to approve faster, trusting that generated code is probably fine. The other is to let an AI reviewer carry the load, as if a model saying “looks good” were evidence of correctness. A model’s review is another opinion. It is useful, but it proves nothing.
This programme puts the evidence somewhere else: in measured detection rates, in tests that fail when behaviour breaks, and in release controls that do not depend on anyone’s vigilance.
What the work involves
Baseline. We measure review and release as they are: pull-request size, time to first review, review rounds, reverts, change-failure rate and defects found after release, and map every route a change takes to production.
Seeded-defect evaluation. In a safe copy of the repository, we take realistic changes and plant known defects of the kinds that matter for your code: logic errors, missing authorisation checks, unhandled failures, concurrency mistakes, and tests that assert the wrong behaviour. We then measure which defects your human reviewers, your AI reviewer and your test suite catch. The result is a detection map for your codebase, not a vendor’s benchmark.
Risk-tiered review. Using the map, we set review tiers. Low-risk changes with strong tests get a lighter review. Changes to authentication, payments, data migrations or shared infrastructure get senior review and smaller size limits. AI-assisted changes are labelled, and their author owns them.
Release acceptance controls. Where the evaluation found gaps, we add controls to CI: required checks on every merge, test-quality checks on changed code (such as mutation testing), pull-request size limits, and staged or flagged release for risky paths with a rehearsed rollback.
This is the method I have published as Vibes Inside Guardrails: rely on mechanical constraints (tests, types, CI gates) rather than on reviewers catching everything by eye.
The signature deliverable
You end with a review workflow, a seeded-defect evaluation and release acceptance controls. Illustrative example of an evaluation summary:
| Seeded defect type | Human review | AI reviewer | Test suite | Control added |
|---|---|---|---|---|
| Off-by-one in pagination | caught / missed | caught / missed | caught / missed | Property test on changed module |
| Missing authorisation check | caught / missed | caught / missed | caught / missed | Senior review tier for auth paths |
| Swallowed error | caught / missed | caught / missed | caught / missed | Lint rule in CI |
| Test asserts current wrong output | caught / missed | caught / missed | n/a | Mutation testing on changed files |
Illustrative example. The format; results come from your own evaluation.
How acceptance is judged
The engineering manager accepts the programme against the baseline: review time and rounds, pull-request size, escaped defects and reverts, measured again after at least two full sprint cycles under the new workflow. The seeded-defect evaluation can be re-run to check that detection improved where controls were added.
Ownership and handover
The review tiers, CI controls and evaluation kit belong to your team. The kit is documented so you can re-run the evaluation after a tool or process change. The rule that AI review is advisory is written into the repository rules and the contributing guide.
Boundaries
If the team has not yet agreed how it uses AI tools, start with AI engineering enablement for software teams. For rules about what coding agents may do, see coding-agent rollout. For Copilot’s own review features, see GitHub Copilot enablement. To make a whole AI-generated application production-ready, see take an AI-generated application into production.