Systematizing AI Code Review

“Write tests before opening a PR.”

I wrote that in AGENTS.md. A PR showed up without tests anyway.

“Use Conventional Comments labels.”

I shared the convention with the team. A week later, I was the only one using labels.

Requests get forgotten. Rules get broken.

But systems keep running.

---

I have failed multiple times at making review culture stick.

I wrote documentation. Nobody read it. I ran a workshop. Forgotten by the following week. I sent Slack reminders. People muted the channel.

Yet every Monday, I kept writing review comments like “fix the formatting” and “reorder the imports.” Thirty minutes, every week.

The person writing those comments felt productive. But what about the person on the receiving end?

“Flagged again.” “Another round of fixes.” Formatting comments are pure friction for the recipient. A design discussion teaches you something. “Your indentation isn’t 2 spaces” teaches you nothing. You just fix it. The reviewer gets tired. The reviewee gets tired.

And when both sides are tired, what happens?

Reviews get ignored.

PRs sit open for three days. Nobody comments. Someone clicks Approve and merges. Review becomes a formality. This is the reality for many teams.

The turning point came when I introduced an AI review tool.

The moment CodeRabbit caught every formatting and linter violation, the review thread changed completely. Instead of “fix the indent,” the conversations became “what about this design?” and “should we keep adding this pattern?” Something shifted.

It felt like installing a traffic light so the officer directing traffic could go back to actual police work.

---

This book is about turning code review from a “request” into a “system.”

Hooks enforce linting. AI handles first-pass review. Humans focus on design and direction.

The approach is straightforward. File structures, configurations, and pipelines are explained at copy-paste granularity.

Some familiarity with harness engineering fundamentals (AGENTS.md, hooks, feedback loops) and basic code review concepts will make the book easier to follow.

Code review is the heartbeat of the harness. When the heart stops, everything else is affected.

Continue this chapter on Kindle →

The 6-Layer Harness Model

Harness engineering designs the operating environment for AI agents across six layers.

Code review sits at the core of Layer 3: Quality Verification.

Think of a house: Layer 1 is the blueprint, Layer 2 is the construction procedure, Layer 3 is the building inspection. Nobody wants to live in a house that was never inspected.

Three Sublayers of the Quality Verification Layer

Breaking the quality verification layer down further reveals three sublayers.

Sublayer	Owner	Speed	Accuracy
Automated gates	hooks / CI	Seconds	Mechanical precision
AI review	CodeRabbit / Copilot	Minutes	Pattern recognition
Human review	Team members	Hours	Design judgment and direction

Speed decreases as you go up, but judgment quality increases. It works like a restaurant kitchen: the dishwasher (automated gate) is the fastest, the sous chef (AI) handles quality checks, and the head chef (human) decides the direction of the flavor.

”Almost Every Time” vs. “Every Single Time”

A SmartScope article nails the point:

Writing “run the linter” in CLAUDE.md vs. enforcing linter execution with a hook is the difference between “almost every time” and “every single time.”

The gap between “almost every time” and “every single time” is wider than it looks. A 90% compliance rate means 1 in 10 PRs ships broken code. With 100 PRs a month, that is 10 uninspected deployments to production.

The same applies to code review.

Method	Compliance rate
Write “write tests before PR” in AGENTS.md	80-90%
Enforce test execution via pre-commit hook	100%
Write “use Conventional Comments” in AGENTS.md	50-70%
Add label reminder in PR template	70-80%

Enforcement increases in this order: request, template, hook, CI. This book designs what to enforce and what to keep optional at each layer.

Why Not Enforce Everything?

You might think “why not enforce everything in CI?” But enforcing everything kills development speed.

Just as security has “the convenience vs. security tradeoff,” reviews have “the enforcement vs. velocity tradeoff.” What matters is drawing the line between what to enforce and what to recommend.

This book’s approach:

• Enforce: formatting, linting, type checks, tests — things machines can judge
• Recommend: Conventional Comments labels, PR size limits — things involving human judgment

File Structure Overview

Here is the file structure of the harness review system this book builds:

Starting from the next chapter, we will design each of the three sublayers in order.

Continue this chapter on Kindle →

In the previous chapter, we established that code review belongs in Layer 3 (Quality Verification) of the harness. This chapter breaks that layer into three sublayers. This structure is the backbone of the entire book.

Dividing Roles Across Three Layers

The biggest problem with code review is trying to have humans do everything.

Style checks, linter violations, type errors, test coverage, security vulnerabilities, design decisions, directional discussions. When one reviewer handles all of that in a single PR, review fatigue is inevitable.

The 3-layer model divides the labor.

Layer 1: Automated Gates (No Human Involvement)

Anything caught here never reaches the PR. It is resolved before a reviewer ever sees it.

Principle: if a machine can judge it, let the machine handle it.

Layer 2: AI Review (Pattern Recognition)

What AI can find:

• N+1 query problems
• SQL injection risks
• Unused imports/variables
• Insufficient test coverage
• Naming convention violations

Principle: if a pattern can detect it, let AI handle it.

Layer 3: Human Review (Judgment and Direction)

Principle: humans should only handle what requires judgment.

The Effect of Three Layers

Because humans can focus on judgment, the quality of reviews also improves.

Responsibility Boundaries per Layer

Issue type	Layer 1 (auto)	Layer 2 (AI)	Layer 3 (human)
Formatting	O
Linter violations	O
Type errors	O
Test failures	O
N+1 problems		O
Security vulnerabilities		O
Naming improvements		O
Design decisions			O
Business logic			O
Directional editing			O

When the boundaries are clear, each layer can focus on its own job.

When Human Review Is Not Needed

Here is an important point: not every PR needs to reach Layer 3.

Bug fixes, dependency updates, and simple ticket work that require no design decisions or business logic validation can be completed with just Layer 1 (automated gates) and Layer 2 (AI review).

PR type	Required layers
Formatting fix	Layer 1 only
Dependency update (Dependabot)	Layer 1 + Layer 2
Bug fix (clear cause)	Layer 1 + Layer 2
Refactoring (small)	Layer 1 + Layer 2
New feature	Layer 1 + Layer 2 + Layer 3
Architecture change	Layer 1 + Layer 2 + Layer 3
New pattern introduction	Layer 1 + Layer 2 + Layer 3

Human review is only needed for PRs that involve judgment. If you do not make this explicit, you end up requiring human review on every PR, and review becomes a bottleneck again.

Continue this chapter on Kindle →