An AI "kill switch" that a third party can verify may need to be mandatory for companies, according to Jack Clark, one of seven co-founders of Anthropic. He told the BBC that a way to fully shut off AI software if it becomes too dangerous is something society "might want to eventually pass rules around."

Clark said most labs, including Anthropic, already have ways to "pull the plug," but lawmakers may need to enforce having one. He added that specifics around kill switch requirements and verification should be part of "the larger policy conversation" around AI.

The comments come amid ongoing debate over AI safety. Last week, a post from a researcher who left Anthropic over concerns AI could wipe out humanity went viral. In response, Anthropic scientist Evan Hubinger said he personally put the chance of human extinction from AI at ">10% within the next decade." Computer scientist and Nobel winner Geoffrey Hinton told the BBC on Friday that a 10% chance of AI killing all humans was "not unreasonable."

Not everyone shares those fears. Clement Delangue, leader of developer platform Hugging Face, said asking about AI extinction risk was like asking your AC guy about climate change. George Arison, leader of Grindr, said such fears were being used by companies to support their business plans.

In the US, lawmakers have put forward a Kill Switch Act that would require companies to have a way to shut down problematic AI tools and give certain agencies power to demand a tool be turned off or limited. President Donald Trump has rejected slowing AI, saying on social media that "AI taking over the World, destroying Humanity, and all other things bad, is a HOAX." In the UK, the government recently rejected creating a kill switch, saying it "would not prevent them being developed or misused elsewhere."

Anthropic, formed in 2021 by former OpenAI employees, makes the Claude chatbot. CEO Dario Amodei over the weekend called for the pace of AI development to slow and be more closely monitored, though some have questioned the motivations behind this. He added that any action should be done "without sacrificing commercial advantage."

Source: BBC News