# Saving 80+% on code reviews using AgentRun

We tuned an open-source code review agent on our own pull requests, catching real bugs for 18 cents a review.

Canonical: https://agentrun.ai/blog/saving-80-percent-on-code-reviews-using-agentrun
Published: 2026-11-12T20:00:00.000Z
Author: Miguel Rios Berrios

*Our code review bill went up 36x this year. So we tuned an agent on our own pull requests, running on a model that
costs almost nothing, until it caught more bugs than the reviewer we were paying for. It's open source, and you can use
it today.*


Last week I sat down with our [Greptile](https://www.greptile.com/) invoices. A year ago we paid $180 a month. In October we paid $6,451. This
month we're at $7,018 with a week to go, on pace for about $9K.

![Our Greptile bill: $180 in December up to $6,451 in October and $7,018 so far in November](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/0-bill.png)

Two things happened. In March, Greptile changed its pricing: a seat used to be a flat $30, and now it covers 50 reviews,
with every review after that at $1. Then our volume took off. Our agents open most of our pull requests now, they made
99% of the reviews this month, and every push gets reviewed again.

Greptile is an industry-leading code review product, and it's good. Its own dashboard says it caught 18,000 bugs for us
this year, about $1.20 a bug. That's a fine deal when a person writes ten PRs a week. It's a terrible one when your
agents write a thousand a month.

Every founder watches costs, and this is the one line on our bill that grows with every agent we add. At $9K a month
that's over $100K a year on code review. So I asked the obvious question: is this worth it, and can we do it ourselves?
I gave my agent one bar: better than Greptile on our codebase, and make it fucking cheap.

Cheap meant [particle-1](https://particle.ai/blog/introducing-particle-1). Particle released it two weeks ago: it's
DeepSeek V4.1 Flash, post-trained with reinforcement learning to finish coding tasks in fewer tokens. Particle reports
about 45% fewer tokens on average, with pass rates about the same as the base model. On [OpenRouter](https://openrouter.ai/) it costs 5 cents per
million input tokens. The whole question was whether a model that cheap could review code worth a damn.

## It can

What we built is one parent agent and eight particle-1 reviewers on a read-only checkout of the PR. The parent checks
every finding against the code and posts one review. It ships as an [AgentRun](https://agentrun.ai/) template, and here's how it did on the 30
newest PRs in our repo, none of which it saw while we tuned it:

![Real bugs caught on 30 PRs it never saw: Greptile 26%, AgentRun 42%](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/1-holdout.png)

![Serious bugs caught out of 8: Greptile 1, AgentRun 5](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/2-serious.png)

1.6 times the real bugs, and 5 of the 8 serious ones to Greptile's 1. For about 18 cents a review, 82% less:

![Cost per review: Greptile $1.00, AgentRun $0.18](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/0-unit.png)

This month's 7,676 reviews would have cost us about $1,400 at that price, instead of the $7,000 and counting we're
paying.

Greptile is pickier. Three of every four things it posts are real, and for us it's closer to one in three. I'd rather
read a wrong finding than miss a real P1, but that's the next thing to fix.

## How it got there

It didn't start at 42%. Our first version read PRs through the GitHub API and caught 4% of the bugs. Then our
self-improving pipeline took over: run the template, grade every finding blind, read where it lost bugs, change one
thing, and keep the change only if it catches more. In about a day and a half:

![Real bugs caught on the 20 PRs we tuned on: API only 4%, + shell 10%, + think 17%, deep 19%, 5 wide 42%, 5 + 3 45%, v1 41%, v2 49%, v3 45%, v3 + 2 48%](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/8-climb.png)

Not every step was a win. v3 cut the noise and lost serious bugs, so v2 is what ships.

## Built for our code, ready for yours

This isn't meant to replace the code review companies. Greptile and the others are building great products for
everyone. We wanted a reviewer that's great on our code, and a template solid enough that anyone can take it and tune
it for theirs.

So we tried it on other people's code too. On [Martian's](https://withmartian.com) 50 open-source PRs, re-judged with
the same judge for every tool, it ties for the most bugs found:

![Bugs found on Martian's benchmark: AgentRun 64%, Copilot 64%, Cubic 63%, Augment 63%, CodeRabbit 61%, Qodo 60%, Greptile 48%, Claude Code 44%](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/6-martian.png)

On ReviewBench's 219 PRs from 187 repos, our first version is precise, 88% of what it posts is real, but it still
misses a lot of what's in the answer key. That's what you'd expect from a reviewer tuned on one repo, and it's why our
self-improving pipeline tunes on your code, not ours. We'll keep both benchmarks as a check we never train on.

Every codebase is different, so you'll want to tune it for yours. For us that was more than worth it: about $18 of
reviews per round, against a bill heading for six figures a year.

## Run your own

You don't need us to host this. AgentRun is open source, and the reviewer is a template in it.

1. Start AgentRun. `npx @parcha/agentrun` runs it on your machine, or hit the Deploy to Render button in the
   [README](https://github.com/Parcha-ai/agentrun) and you have your own server for about $27.50 a month.
2. Add an OpenRouter key and apply the **code-review** template to your repo. It sets up its own pull request trigger,
   and from then on every PR gets a review from your own server, on your own key, for about 18 cents.
3. Add a `docs/CODEREVIEW.md` with what matters in your code and what to never flag, and turn on `"learning": true` so
   it keeps notes from every review.

Want it tuned to your code before our pipeline ships? [Tune a template on your own code](https://github.com/Parcha-ai/agentrun/blob/main/docs/tutorials/tune-a-template.md)
is the same loop, step by step.

## Next: it tunes itself, for you

That climb was our self-improving pipeline at work, and an agent ran all of it. Nobody tuned this reviewer by hand.

We're shipping that pipeline in [console.agentrun.ai](https://console.agentrun.ai) for paying customers next. Point it
at your repo's merged PRs, and it tunes the reviewer to your code the way it tuned ours, and keeps tuning as your code
changes. AgentRun's learning already keeps notes from every review. The pipeline adds the part a single review can't
know: which of its calls were wrong.

We're also making it even cheaper. Not every PR needs eight reviewers. A docs fix doesn't need a review at all, a
small change needs one quick look, and a change to billing code needs someone who thinks about money. So next, the
review becomes a workflow, written in AgentRun's workflow DSL. A Jev step reads every PR first and routes it, and on
the risky ones it adds the specialist reviewers that PR needs:

![A workflow routes each pull request with Jev: no review for docs, config and renames at $0; one quick reviewer for small, simple changes at about 2 cents; a full review with 5 quick and 3 deep reviewers at about 18 cents, plus specialist reviewers for concurrency, money, persistence and untrusted input at about 3 cents each](/blog/imported-writing/saving-80-percent-on-code-reviews-using-agentrun/9-router.png)

We already tested the routing part. Jev read 223 of our PRs for less than 10 cents in total, and scored against our
answer keys, a simple rule on its answers would have sent about 1 in 7 PRs to a single quick reviewer without missing a
serious bug. It's also sharp at spotting the rare, high-stakes stuff (concurrency, money, persistence, untrusted
input), so the specialists only show up where they matter. By our estimate, that gets the average review down to
about 10 to 13 cents. And because it's a workflow, you can watch every PR go through it in the console, see why Jev
sent it where it did, and change the rules yourself.

Try it on your PRs and tell me where it's wrong.

**Notes.** Reviews ran on the AgentRun code-review template with particle-1 through OpenRouter, each reviewer on a
read-only checkout holding only the PR's history. On our PRs, Opus 5.5 graded every finding blind, and every real bug
anyone found went into one answer key (78 bugs, 8 serious, on the 30-PR holdout); ours count only where the judge called
them real, and Greptile gets credit wherever it described the same bug. Martian scores are re-judged with Opus 5.5
through Martian's own pipeline, so they differ from its site. ReviewBench is our first version, self-run for one round
with the board's own judge, never submitted; Greptile numbers come from our
own invoices and dashboard. Costs are what OpenRouter billed.