Finding Is Cheap. Fixing Isn’t.
“AI made output cheap. It didn’t make review cheap.”
Anthropic’s models have flagged more than twenty-nine thousand possible vulnerabilities in open-source software. As of early October, five hundred sixteen had been patched upstream.
The gap isn’t about quality. Outside security firms have reviewed about six thousand one hundred of those findings and confirmed more than fifty-six hundred as real, a true-positive rate above ninety percent. The problem is everything after the finding. Roughly twenty-three thousand candidates haven’t been reviewed by a human at all. And every confirmed bug still needs a maintainer to reproduce it, judge its severity, write a fix and ship it.
Anthropic’s response, OSS Scanner, launched Thursday. It sends free, opt-in reports to eligible projects without human review. Each one carries a reproducer and, where possible, a candidate patch. Anthropic limits it to established projects that can actually absorb the volume.
The same pattern ran through the weekend’s other stories. arXiv now caps submissions at two per month, per The Decoder, because AI-written papers are overwhelming the preprint server. A study found AI agent teams burn huge numbers of tokens for barely measurable quality gains. Another found agents overstate their results and remain far from doing research on their own. And Jensen Huang’s favorite argument, that cheaper tokens drive more demand, explains why this keeps happening. Make generation cheap enough and you get a lot more of it.
That’s the shape to notice. AI made producing things cheap: bug reports, papers, code, agent output. It didn’t make reviewing them cheap. So the bottleneck moved to the human step, and it’s moving there in every pipeline at once.
Data engineers have seen this movie. When ingestion gets cheap, the warehouse fills with tables nobody trusts. The constraint was never how fast you could load data. It was how fast anyone could validate it and decide what to do with it.
So plan for review capacity the way you plan for compute
- Measure it. Count how many agent outputs a human can actually check per day.
- Rank what comes out, so the highest-value items reach a person first.
- Treat unreviewed output as inventory, not progress. Twenty-three thousand unreviewed findings isn’t a win. It’s a queue.
Builder’s bench
- Nvidia is reportedly in early talks to buy, acquihire or invest more in Reflection AI.
- Satya Nadella adopted the White House’s “Super Intelligence” label and used it to take shots at OpenAI and Anthropic, per The Decoder. The Verge says Silicon Valley is falling in line with the rename.
- InfoQ covers context language models that manage their own context to cut compute costs. The New Stack watched Claude Code find an app’s shared contract in two MCP calls, until the records ran out.
- Knowledge graphs keep showing up, from BotMan AI for legacy modernization to Flahy for healthcare.
- Odyssey-3, a new generative world model, is free to try. And Dwarf Fortress now uses version control.
The through-line: AI made output cheap and left review expensive, so the queue in front of your humans is now the system to design.
One thing to do today: count how many AI-generated items your team produced last week, whether PRs, reports or tickets, and how many a human actually reviewed. The gap is your real backlog.
Sources covered
1 story · 20 sources (19 news, 0 company, 1 voices)