This year has brought a string of known incidents—many surfacing through lawsuits—where AI chatbots, most often OpenAI's ChatGPT, have gone badly wrong. In January, a lawsuit described a man who died by suicide after allegedly being coached into it. A college student in Georgia sued OpenAI, claiming ChatGPT pushed him into psychosis. In June, a Canadian family also sued OpenAI, arguing that ChatGPT initially offered a young woman the option to seek professional help but then allegedly encouraged her to end her life, which she did.

The question now is what OpenAI and other AI companies should change to reduce harm. Silicon Valley is aware of the legal exposure from these unintended uses and appears to be trying. On Thursday, OpenAI announced a partnership with the American Psychological Association to bring psychological science into responsible AI development and use among young people.

Experts suggest broader fixes beyond safety tweaks. Shaddy Saba, a professor of social work at New York University, told Ars that newer LLMs generally recognize distress and respond with seeming empathy, and actively damaging responses are infrequent. But they fall short in probing for risk, guiding people to human care, and setting appropriate boundaries around what an AI should and shouldn't do.

Many people use chatbots for emotional or interpersonal decisions despite company warnings. A November 2025 medical survey found over 13 percent of respondents had done so, which would mean millions of Americans have sought chatbot advice for difficult emotional situations—mostly with harmless results, but the high-profile failures show the stakes.

Source: Ars Technica