Facing a bottleneck of thousands of monthly merge requests, Cloudflare engineers built an automated review system that now handles their entire code submission pipeline. The architecture, detailed by engineer Ryan Skidmore, embeds directly into their GitLab CI. On each request, a coordinator agent dispatches up to seven specialized reviewers for security, performance, and internal standards. The system processed 131,246 reviews last month, with a median runtime under four minutes.

Engineering trade-offs are central to the design. The system applies a risk-tiered approach: trivial changes trigger only two agents, while major diffs get the full panel. It first strips noise from generated code and minified files. A plugin architecture allows swaps for the version control system, AI models, and rule sets. For model routing, they use their own AI Gateway with configured fallback chains, defaulting to cheaper models like Sonnet 4.6 for routine analysis and reserving costlier ones like Claude Opus for coordination.

The technical execution avoids common pitfalls. Agents run as separate Bun.spawn processes, with JSONL streaming to prevent memory issues. Prompts are explicitly scoped with 'What to Flag' and 'What NOT to Flag' sections to reduce hallucinations. The coordinator agent deduplicates and categorizes findings before posting a single comment, biasing toward approval unless critical issues or repeated warning patterns are detected. A 'break glass' human override was used in only 0.6% of runs.

Underpinning this is Cloudflare's internal stack. AI Gateway routed 24.77 billion tokens for these reviews last month, with Workers AI providing inference at roughly a quarter of the cost of proprietary APIs. An 'Agent Memory' layer allows the system to recall past review decisions, reducing repetitive false positives. The result is a production system that reviews every merge request, costing about a dollar per review on average, while weekly submission volume has doubled.

Source: Webpronews