Anthropic's S-1 filing with the SEC, published Monday ahead of its widely anticipated initial public offering, spends considerable space on the company's soaring revenues and costs-but also on risks that could bring it down.

Across 93 pages and more than 70,000 words, the "Risk Factors" section lists hundreds of potential problems. Most are standard business worries about competition, costs, customer acquisition and regulation. Some stand out.

Anthropic acknowledges concerns about risks that could "pose catastrophic or existential risks to humanity," raised both inside the company and by researchers, regulators, policymakers and industry participants. The specific worry, the company says, is not any current model but near-future ones that could "substantially automate AI research and development" on their own. If allowed to operate with high autonomy, such systems could advance to a point that "accelerate[s] AI progress beyond our ability to evaluate and safeguard new model," Anthropic warns.

Source: Ars Technica