Anthropic has given outside evaluators "employee-like access" to probe the risks of its models, acting on a pledge to tighten safeguards over fears the technology could spiral catastrophically out of control.

METR, a nonprofit that has worked with OpenAI, Anthropic, Google and Meta, now has a team stationed inside the Claude maker's offices, with access to its hardware and internal systems.

Third-party evaluation has become one of the main concrete steps labs have floated as concern grows over the dangers more capable AI systems could pose to the economy, cyber security and humanity at large.

Anthropic chief executive Dario Amodei found a rare point of agreement with rivals Sam Altman and Elon Musk in September, when the OpenAI and SpaceX bosses backed his proposal to embed external auditors in leading labs. Thursday's announcement is the first step toward carrying out those pledges. OpenAI has yet to detail plans to bring in evaluators after Altman called Amodei's proposals a good idea.

"As models have become more powerful, they've required stronger safeguards and more challenging risk assessment," Anthropic said. "We want to be publicly accountable for the decisions we're making. But we can't publicly share all the information relevant to our safety work."

Critics have warned the arrangements could breed conflicts of interest, citing problems in the accounting sector, and argue stronger government regulation is needed. Anthropic countered that public disclosure could aid malicious actors or competitors, and said its evaluators will work much like bank supervision, where regulators assign resident examiners inside the largest banks.

Source: Ars Technica