What I think reviewers should look for in agent-written code

Bighorn sheep grazing among scattered rocks on an alpine tundra slope in Colorado

Code review used to lean on a quiet signal that nobody talked about. Code that looked right usually was. A person who produced clean, well-named code usually understood the problem, so reviewers could spend less attention on clean changes and more on messy ones.

An agent produces code that looks right every time. The signal is gone, and review has to change. Here is what I look for.

Does it match the ask, and only the ask

The first thing I check is scope. Agents are eager. Asked to fix a bug, they fix it, tidy the surrounding function, rename a variable, and add a helper nobody asked for. Each change is defensible, and together they turn a ten-line review into a two-hundred-line one with the fix hidden inside. I think a reviewer should hold agent-written changes to a stricter scope than human ones, not a looser one. If the change does more than the task, the extra parts are unreviewed by anyone, including the author.

Do the tests test anything

Agents write tests readily, and the tests often pass, which proves less than it seems. I look at what the test would catch. A test that asserts the function returns what it currently returns is a snapshot, not a test. A test that mocks the thing being tested is decoration. The question is whether the test would fail if the behavior were wrong, and that takes reading the test, not counting it.

What could the agent not see

An agent sees the repository. It does not see the shape of the production data, the downstream system that chokes on nulls, or the reason a strange-looking check was added three years ago. I think the reviewer's highest-value contribution is now exactly the things that are not in the code.

What was weakened or duplicated

Two patterns I have learned to look for. The first is a check that got loosened to make a test pass, a type widened, a validation removed, a timeout extended. The second is a new helper that duplicates one the agent did not find. Both are easy to miss in a large diff and both compound.

Does the explanation match the diff

Agents write good pull request descriptions, and I think that is a hazard. With a person, the description and the code were produced by the same understanding. With an agent, a model can describe what it meant to do rather than what it did. I read the diff first and the description second, and when they disagree, the diff is the truth.

The author still owns it

The person who opened the pull request owns every line in it, however it was produced, and I think a team should say so plainly. The agent is a tool the author used, and the review is a second pair of eyes on the author's work, as it always was. What changed is what those eyes look for.

Photo source: https://photos.robertstowe.com/colorado