Technical Paper · July 2026

Argus: A Systems-Aware Agentic Code Review Harness

Dan Costanza @ Redesign Health, July 2026

Abstract

The AI community largely agrees that agentic engineering has shifted the hard problem from writing code to verifying it. Agents now enable reasonable-looking code to ship faster than any review process built for human-authored code can process. In addition to the volume, agent-written code is generally locally fluent and globally inconsistent, and thus difficult to review. The bugs it produces tend to live between files: in contracts, deploy ordering, and invariants the change touches but doesn’t sit inside.

This paper describes the system we built in response. Argus reviews pull requests by decomposing each change into the flows it participates in, dispatching repository-aware specialist agents in parallel against those flows, validating findings before surfacing them, and persisting every round into a corpus that feeds back into the harness on a weekly cadence. The architecture borrows from the same ideas now showing up in the broader agentic-harness conversation: stochastic and deterministic hooks composed deliberately, just-in-time context retrieval, structured outputs, and explicit coverage accounting.

We have tested existing code review systems including Greptile and Claude Code Review / Ultrareview and like others have found them to struggle on complex reviews. Building off those learnings, we developed a self-improving agentic harness that addresses the issues their current (fast improving!) versions face.

The current production system has run several thousand production review cycles to date; across a representative 60-day sample (the 218 cycles analyzed in §5), it averages 3.3 rounds, ~35 minutes, and ~$18 per PR. It surfaces roughly 5× more developer-validated blockers than Greptile on the same PRs. While these signals indicate strong performance, we have not validated against formal benchmarks like Augment’s precision/recall framework and Martian’s Code Review Bench, but have seen significant improvement in real-world outcomes.

We built this internally out of necessity, not out of any desire to maintain code review systems. We hope that any modest contributions we are able to make to the frontier of agentic review will accelerate our ability to decommission Argus and switch to an externally maintained system that performs at or above this standard.

1. Established Baseline: Code Review Is Today’s Binding Constraint

Agentic coding is a paradigm shift in productivity, but becomes a debt-issuance machine without code review that scales with the new volume of code.

The volume side of this is well understood. Our annual run rate is roughly 2,750 merged PRs and ~8.8 million lines of code added across five active human contributors. Call it a completely untenable ~14 FTE-years of careful review work if done by hand.

And the harder part is the subtlety of the bugs. Agent-generated code is generally locally fluent and correct in isolation. The bugs that result tend to look like this:

• a session leak that only manifests across an await boundary three calls deep;
• a utility quietly re-implemented inside this PR while a battle-tested version of the same thing already exists three directories over;
• a migration that needs to deploy before the code that depends on it, but didn’t;

None of these bugs is visible in any single file. The local code is correct. The bug is in the relationship between files, and that is precisely the surface that file-by-file review struggles to see.

The review problem under agentic generation is therefore not throughput. It is semantic coverage: for every change, has the system actually examined the flows, contracts, and cross-file invariants the change participates in. The hard part is preserving that coverage within fixed context windows, by decomposing review across agents. The naive thing to want is bigger context windows; the actually-useful thing is a system that knows how to decompose systems and evaluate code in a logical manner, not just brute force it.

Two failure modes for less sophisticated approaches drop out of this framing immediately.

Diff-local review breaks down. Tools that read each changed file top-to-bottom, even with neighbor retrieval, can’t evaluate complex interactions, such as whether the migration in this PR deploys before the code that depends on it, or whether the new backend response matches the existing frontend contract. This is a structural limitation of diff-anchored review, not a prompt-quality problem.

Single-context review hits the wall. A reviewer with a fixed context window cannot hold a 75 file PR plus the surrounding code it interacts with: the diff, the consumers of the changed API, the migration that has to land first, the existing utility this code is duplicating, the test fixtures that need to update. The naive workaround, chunking the diff into file groups and reviewing each group independently, destroys exactly the cross-cutting checks that matter most.

The right approach is to evaluate the code like a human reviewer:

1. Find logical entry points (“user sends input i to api/v1/…”)
2. Follow a specific action through all its steps
3. Evaluate the complex chain end-to-end through and across files

2. Landscape and Prior Art

The market is converging quickly on this architecture. We are participating in an emerging pattern, not discovering one.

Cloudflare published “Orchestrating AI Code Review at scale” on April 20, 2026, two weeks before this paper. They describe a CI-native pipeline with up to seven specialist reviewers (security, performance, code quality, documentation, release management, compliance) feeding a coordinator agent that handles deduplication, re-categorization, and a “reasonableness filter” to drop nits and false positives. Reviewers emit structured XML with critical/warning/suggestion severity. They tier by risk: trivial PRs (≤10 lines, ≤20 files) run a downgraded path with only the coordinator and a generalist. Cloudflare’s stated motivation matches ours.

Anthropic’s Claude Code Review describes a fleet of specialized agents reviewing PRs in full-codebase context for logic errors, security issues, edge cases, and regressions. While the system design is directionally correct, we and others have found its performance thus far to be sub-standard.

Qodo positions around multi-agent review, rule enforcement, multi-repo context, governance, and learning from PR history.

Greptile, CodeRabbit, Augment Code, Cursor BugBot, GitHub Copilot Review, and others are commercially active in adjacent space. Each takes a different cut at the diff-anchored vs. repo-aware tradeoff.

Augment Code published a benchmark of seven AI code review tools in December 2025 using a precision/recall/F-score methodology against gold-set ground truth on 50 PRs across five open-source projects. Augment 65/55/59, Cursor BugBot 60/41/49, Greptile 45/45/45, Codex 68/29/41, CodeRabbit 36/43/39, Claude Code 23/51/31, GitHub Copilot 20/34/25 (precision / recall / F).

Martian’s Code Review Bench v0 (February 2026) is the methodologically sharpest piece of work we’ve found in the space. They argue, and demonstrate, that benchmark gold sets are systematically incomplete: comments scored as false positives often turn out to be real issues the gold set missed. They report that “no tool found more than 63% of known issues” on their first round.

3. Design Principles

Argus is built around seven principles. They are the durable part of the system; the specific prompt graph and model choices below them will change.

1. Review execution flows, not files. The unit of review is an action plus its transitive calls, mutations, contracts, deploy dependencies, and consumers, not a changed file read top to bottom.
2. Use just-in-time repository context. No agent preloads the whole repo. The planner identifies which flows matter; specialists then use lightweight references and tools (read, search, glob, symbol lookup) to dynamically load the right context at runtime.
3. Decompose review, then verify coverage. Parallel agents create a coverage problem — the decomposition can drop changed files entirely. The system needs to prove, or at least check, that every changed line was examined by at least one reviewer.
4. Separate finding generation from severity assignment. Specialist agents identify candidate issues. A later writer pass deduplicates, calibrates, and assigns severity holistically.
5. Validate blockers before surfacing them. A dedicated validation pass re-confirms each critical finding against the actual diff before the verdict reaches the engineer. This is the system’s primary anti-hallucination mechanism.
6. Track finding identity across rounds. The system knows whether a prior issue was resolved, regressed, dismissed by a reviewer, or still open.
7. Convert review feedback into harness improvements. Recurring review findings are signals that something in the development environment can be made deterministic, codified, or pushed upstream into the writing step.

4. Architecture

4.1 Pipeline Overview

Argus is a graph orchestrator with parallel agent fan-out. Each stage is either a single LLM call or a tool-using agent session.

Stage 0: Feedback verifier (round 2+). Before any new review begins, the system reads findings from the last round from durable storage and checks each against the new diff, marking it resolved, regressed, or still open.

Stage 1: Planner. Reads the code changes / PR diff, description, and relevant repo conventions. Produces a ReviewPlan containing system-level review slices, a file manifest, cross-cutting concerns to investigate, and which specialists to dispatch per slice.

Stage 2a: System reviewers (parallel). One reviewer per slice. Each runs as a standalone session with read, search, and glob tools over the entire repository, mapping the flow end to end, across files.

Stage 2b: Specialists (parallel with 2a). The planner flags specialists_needed per slice based on file patterns.

Stage 2c: Cross-cutting reviewer (parallel with 2a/2b). A dedicated reviewer focused only on across-flow / multi-file path tracing, migration vs. code deploy ordering, frontend/backend contract drift, session lifecycle across await boundaries, conditional execution paths, re-invention of internal utilities.

Stage 3: Coverage check. Given the file manifest and all collected findings, returns whether every changed line was included in a reviewed flow. If gaps exist, dispatches 1–2 targeted reviewers to fill them.

Stage 4: Writer. Consolidates all findings, assigns severity (critical / suggestion / nit), produces the verdict and risk level, and emits a formatted review comment.

Stage 5a: Blocking validator. For each critical finding, a dedicated pass re-confirms the issue against the actual diff. False positives are downgraded before the verdict reaches the engineer.

Stage 5b: Persisting findings. Every finding from the round is written to durable storage alongside the verdict, full agent telemetry, and the final review comment. An external weekly process reviews aggregated findings to surface recurring patterns and propose refinements to specialist prompts and planner heuristics.

5. Operating Results

Measured end-to-end across 218 production review cycles in the 60 days before publication, bucketed by total PR lines changed (additions + deletions). All values are averages.

5.1 Comparison vs. Greptile on the Same PRs

We heavily use Greptile and Claude review, which continue to be powerful and useful tools. However in our testing we find that Argus adds significant additional value in addition to what they accomplish. 154 of the 218 PRs above were also reviewed by Greptile.

5.2 How to Read This Comparison

The comparison above is an operational signal, not a benchmark. The cumulative blockers we count for Argus are developer-validated: every blocker flows into the autonomous fix loop, where the engineer either accepts it or explicitly dismisses it with a reason that persists across rounds. Findings dismissed as false positives drop out of the count.

6. From Review Pipeline to Learning Harness

Argus is already more than a review pipeline. Stage 5b persists every finding, dismissal, fix, and round of agent telemetry; a weekly external review aggregates that corpus and uses it to improve future coding agents by updates to their prompting (claude.md, etc) and tooling. Additionally, the job auto-refines specialist prompts and planner heuristics to ensure frequent error types are caught earlier in the review process.

Reviewing PRs faster matters. Stopping the same kind of bug from recurring matters more.

7. Closing Thoughts

We believe that code review will continue to be one of the most critical problems in agentic coding for the foreseeable future, and that over time best-in-class shared tools from Greptile, Anthropic and many others will dominate the space.

In the meantime, we believe we have made some significant improvements to review performance and wish to contribute them back to the broader AI research community.

To do this, we took a structurally different approach than existing systems. Through careful orchestration, we drove the agentic reviewers to approach their review processes similar to how human reviewers do. Humans don’t just read files and look for errors. They follow logical paths through and across files, ensuring continuity of logic and data.

We hope that in sharing this approach, others will take it on, extend it, improve it, and help move us towards a world where building amazing technology to solve practical problems becomes faster, safer and more reliable.

Appendix A: An Example Review Round

The PR. A backend refactor that promotes a shared helper from a single project’s code into a common library so a second project can also use it. Three files changed, just under 400 new lines, with a small backward-compatibility shim left behind in the original location. The kind of change that looks innocuous on the surface — a function moved, an import updated, a test ported — but quietly modifies a SQL query at the heart of a connection-matching pipeline.

The result. One round of review, ~9 minutes, $4 in model spend. Verdict: blocking, with one critical issue and six suggestions.

What Flows the Planner Flagged

The planner read the diff and identified six distinct flows worth reviewing, each one a different question this kind of PR can fail at:

• The moved helper itself. Did the function still behave correctly after being lifted out of its original home?
• The SQL the helper issues. Does the new query follow the index conventions the rest of the codebase uses?
• The shim left behind in the original location. Will existing callers of the old import path still work?
• The test suite. Do the tests actually exercise the new code path, or only the old shape of the code?
• Security. Are there any new injection or unbounded-input paths?
• Backward compatibility. Do existing tests that monkeypatch the old import path still resolve correctly?

How the Agents Fanned Out

Eight reviewer agents ran in parallel against the repo.

What the Review Found

Critical: silent data loss between Python and SQL. The new helper has a Python function that strips trailing slashes from URL slugs, turning …/in/john-doe/ into john-doe. But the SQL query alongside it does not apply the same normalization. So when the database stores the raw form john-doe/, it never matches the stripped Python form john-doe, and those rows are silently dropped from the result.

This is the kind of finding the architecture is built to surface. Reading the Python file alone, the function is correct. Reading the SQL alone, the query is correct. The bug lives in the contract between them — one side strips, the other doesn’t, and the join silently fails.

The remaining findings covered missing integration tests, canonical query patterns, a missing length guard, a misleading comment, and a stale type-ignore.

Appendix B: Limitations

• The market is moving fast. External tools are improving on a daily cadence. Today’s gap may close.
• More findings is not the same as better review. We do track Argus’s own dismissal rate, accepted-finding rate, and false-positive rate, but this is directional evidence, not a robust benchmark.
• Coverage checks do not prove correctness. Argus’s coverage check verifies every changed line was examined by some reviewer. It does not prove the reviewer reasoned about the right thing.
• LLM reviewers still hallucinate. The blocking validator is a mitigation, not a guarantee.
• Some classes of issues belong in deterministic tools, not in prompts. We try to push these out of the LLM path as we identify them.
• Autonomous fix/re-review loops require human escalation paths. A pure autonomous loop is not appropriate for all PRs.
• The current prompt graph is likely temporary. The architectural principles in §3 are what we expect to survive; the specific node graph in §4 is not.

References

• Cloudflare, “Orchestrating AI Code Review at scale,” April 20, 2026.
• Augment Code, “We benchmarked 7 AI code review tools on real-world PRs,” December 11, 2025.
• Martian, “Code Review Bench: Towards Billion Dollar Benchmarks,” February 26, 2026.
• Anthropic, “Code Review for Claude Code.”
• Anthropic, “Effective context engineering for AI agents.”