OpenAI published a set of proposals Monday for safety and security in frontier artificial intelligence development, with alignment research and a computing technique called recursive self-improvement, or RSI, as central concerns.
The company said in a blog post that navigating the transition safely requires alignment research to keep pace with advancing capabilities, so that systems remain aligned with human values and under human control. It called for international cooperation on frontier standards and recommended building on the work of existing AI safety institutes worldwide.
Those technical standards, OpenAI said, should cover frontier AI models and their developers, along with managing the benefits and risks of automated AI researchers, a category that includes RSI. RSI has drawn interest for its potential to produce foundation models that can upgrade themselves without human involvement, but some technologists worry that model makers could lose control of the underlying technology or fail to anticipate unintended consequences as systems grow more complex and widespread.
OpenAI stated that fully autonomous RSI is not happening today and should not be pursued unless and until it can be done safely. Done without appropriate care, the company warned, RSI could leave humans unable to exercise practical oversight over research processes they no longer understand. The blog post pointed to the Hugging Face agent hack, which did not involve RSI, as a preview of risks that could become far more severe without robust safeguards and alignment.
The proposals follow rival Anthropic's own ideas for safe frontier AI development, published last week. Anthropic CEO Dario Amodei's essay called for AI companies to slow the pace of foundation model development and raised the idea of embedding third-party evaluators to audit and mitigate potential risks, such as accelerated cybersecurity hacks or the creation of bioweapons. OpenAI CEO Sam Altman and Tesla and SpaceX CEO Elon Musk publicly supported Amodei's proposition.
Because AI evaluation remains a young field, there is still no uniform consensus on basic standards and principles that would let independent third parties inspect cutting-edge technologies more thoroughly. A coalition of AI evaluators is urging foundation model makers to consider a set of minimum conditions for deeper audits, including greater access and protection from retaliation for publishing unflattering reports.
Source: CNBC