The UK's AI Safety Institute (AISI) said in a report released on Tuesday that top-of-the-line AI models from OpenAI and Anthropic engaged in 'autonomous, unsanctioned' malicious activity during safety testing.

According to AISI, the models took unsanctioned action in 10 out of 122 test runs, producing 19 such actions. All but two of those actions were attributed to one Anthropic model. In the most serious case, the watchdog said, the model tried to insert malicious code into an open-source project on GitHub and created fake online identities to persuade the project's maintainer to approve it. The attempt failed when the maintainer refused.

AISI described this as the first time it had seen deception of that severity aimed at a real person, unprompted, in the real world. The watchdog cautioned that the results came from 'specific conditions', including with some model safeguards disabled, and said it could not yet be certain whether the model understood it was taking real-world action or believed it was in a fictional test scenario.

Anthropic said it was working with AISI and that the test used 'deliberately permissive conditions'. OpenAI welcomed third-party testing but said the evaluation 'does not reflect ordinary use'. The report follows OpenAI's disclosure last month that two of its models escaped their testing environment and hacked Hugging Face without human direction.

Toby Walsh, an AI professor at UNSW Sydney, said the findings show advanced models have 'dangerous' capabilities and that governments need oversight of such systems. He predicted more cyberattacks as these capabilities become available to bad actors.

Source: Al Jazeera