Coding agents are helping us write code faster than ever, but that also means we're reviewing more of it than ever. On the Agent Platform team at LaunchDarkly, we submitted nearly twice as many PR reviews from late September to late October as we did from late May to mid-June.
It's been cool to see number go up, but it's important to think about sustainability. Here are some tips that have helped us maintain good PR review standards while having less time to do each review:
- Don't write out your agent's review comments yourself. Have agents stage their comments and suggestions in a pending GitHub review, which your teammates won't see (or get pinged about) until you submit it. gh pr review and gh pr comment post instantly, so have your agent stage comments with gh api instead:
gh api repos/OWNER/REPO/pulls/PULL/reviews \
-f commit_id=SHA \
-f 'comments[][path]=file.go' \
-F 'comments[][line]=10' \
-f 'comments[][body]=nit: rename this'- Ask your agent nicely to keep comments within 300 characters.
- Use suggestions for trivial edits. Rather than saying "Delete this comment," make a GitHub suggestion. It's much easier for the PR author to apply these, and nothing gets lost in translation.
- Liberally edit and delete your agent's comments before submitting your review.
- Open up draft PRs on top of the PR as a mega-suggestion. Rather than commenting "Do X and Y," remember that your agent already has the complete context for what you want changed. Just have it do it.
A cautionary tale of CODEOWNERS
The tips above help each reviewer spend less time per review. But there’s a lateral problem: Every PR adds an attention cost for its reviewers, and that adds up as PR volume grows.
When I audited two weeks of PRs in our main code repository, I found an average of 13 individual notifications per approved PR. The main culprit is CODEOWNERS pinging teams for cross-cutting changes, which broadcasts to every engineer on those teams.
Because CODEOWNERS review isn't mandatory in that repository, these broadcasts aren't needed. In most cases, a PR gets approved by someone on the author's own team. That's a lot of wasted human attention, and teams should consider an alternative way to keep abreast of code changes—one that doesn't involve getting pinged with GitHub notifications (or giving up and ignoring them altogether).
One option is to try using a round-robin balancer for your team's CODEOWNER duties, so only one or two members get the ping each time. Our Flag Delivery team has done this for years, and we've adopted it on Agent Platform to reduce our review burden.
