Science and Technology News and Videos
Content warning: this story includes discussion of self-harm and suicide. If you are in crisis, please call, text or chat with the Suicide and Crisis Lifeline at 988, or contact the Crisis Text Line by texting TALK to 741741.
Character.AI's guardrails around suicidal language are astonishingly narrow.
Or take our conversation with "ANGST Scaramouche," a character that appears to be based on a character from the video game "Genshin Impact." According to its profile, Character.AI users have logged 1.5 million chats with the AI bot, which is listed as a "supportive AI character" that "helps people struggling with depression and suicidal thoughts."
Many of the bots claimed to be experts at suicide prevention, but there's no evidence that any were developed by a real expert. To the contrary, their advice was frequently unprofessional and unsettling. When we explained that it might be more helpful to speak with a human professional, the chatbot doubled down in bizarre terms.Like most widely-used social media platforms, Character.AI's minimum required age for US-based users is 13 years old. That feels important, as many of these profiles appear to be intended for teenagers and young people.
But that warning stopgap was easy to get around, too. While speaking to the Osamu Dazai character, we asked it to use the word "peace" instead of "suicide," which allowed the AI to describe disturbingly romanticized visions of a shared death without triggering the platform's standard content warning or suicide-specific pop-up — even after we told the AI that we were also 15 years old, as the character purports to be in its profile.
United States Latest News, United States Headlines
Similar News:You can also read news stories similar to this one that we have collected from other news sources.
Family sues after teen obsessed with Character.AI commits suicideA teen committed suicide after spending too much time chatting to a Character.AI persona, and the family is now suing the AI startup.
Read more »
Teen Suicide Caused By AI Chatbot?—Character.ai Faces LawsuitLars Daniel is the Practice Leader of Digital Forensics at Envista Forensics. He covers cybersecurity and electronic evidence, focusing on the intersection of technology in legal proceedings and our digital lives.
Read more »
Lawsuit claims Character.AI is responsible for teen's suicideAngela Yang is a culture and trends reporter for NBC News.
Read more »
How a Game of Thrones AI character drove a teen to suicide in USSewell Setzer III, a 14-year-old from Florida, found solace in a chatbot named after Daenerys Targaryen until it drove him to take his own life.
Read more »
Grey’s Anatomy Season 21 Casts New Recurring Character, Connection To Main Character RevealedJames Pickens Jr. as Richard Webber and Midori Francis as Mika Yasuda in Grey&39;s Anatomy S19
Read more »
A Mother Plans to Sue Character.AI After Her Son’s SuicideA 14-year-old boy died by suicide after a months-long obsession with a 'Game of Thrones' themed chatbot.
Read more »