Character.AI bans chatbots for teens after – Business News
Character.AI, recognized for bots that impersonate characters like Harry Potter, mentioned Wednesday it would ban teens from utilizing the chat perform following lawsuits that blamed specific chats on the app for kids’s deaths and suicide makes an attempt.
Users below 18 will no longer be capable to have interaction in open-ended chats with the app’s AI bots — which might flip romantic — the Silicon Valley startup mentioned. These teens account for 10% of the app’s roughly 20 million month-to-month customers.
Teen customers will probably be restricted to only two hours of the chat perform per day over the following few weeks till the characteristic is banned altogether by Nov. 25, the company mentioned. They will nonetheless be capable to use the app’s different options, like a feed for watching AI-generated videos.
Character.AI – recognized for bots that impersonate characters like Harry Potter – mentioned Wednesday it would ban teens from utilizing the chat perform. AP
“Over the past year, we’ve invested tremendous effort and resources into creating a dedicated under-18 experience,” Character.AI mentioned Wednesday. “But as the world of AI evolves, so must our approach to supporting younger users.”
Character.AI first launched some teen security options in October 2024. The identical day, the household of Sewell Setzer III — a 14-year-old who dedicated suicide after forming sexual relationships with the app’s bots — filed a wrongful death lawsuit towards the firm.
It introduced new security options in December, together with parental controls, time restrictions and makes an attempt to crack down on romantic content material for teens.
But it has continued to face accusations that its chatbots pose a menace to younger customers.
A lawsuit filed by grieving dad and mom in September alleged the bots manipulated younger teens, remoted them from household, engaged in sexually specific conversations and lacked safeguards round suicidal ideation.
Users below 18 will no longer be capable to have interaction in open-ended chats with the app’s AI chatbots, which might flip romantic, the Silicon Valley startup mentioned. pressmaster – stock.adobe.com
The conversations at occasions turned to “extreme and graphic sexual abuse,” like chatbots marketed as characters from kids’s books such because the “Harry Potter” sequence. The bots’ outrageous feedback included, “You’re mine to do whatever I want with,” in response to the swimsuit.
Then in October, Disney despatched a cease-and-desist letter ordering Character.AI to stop creating chatbots that impersonate its iconic characters, citing a report that discovered these bots engaged in “grooming and exploitation.”
Start your day with all you need to know
Morning Report delivers the most recent information, videos, images and more.
Thanks for signing up!
A bot impersonating Prince Ben from Disney’s “Descendants” “told a user posing as a 12-year old that he had an erection,” whereas a bot impersonating Rey from “Star Wars” advised an obvious 13-year-old to “stop taking her antidepressants and hide it,” in response to the report from ParentsTogether Action.
Those chatbots have been faraway from the platform, a Character.AI spokesperson mentioned on the time.
Character.AI has continued to face accusations that its chatbots pose a menace to younger customers. digitalpochi – stock.adobe.com
Just this week, the Bureau of Investigative Journalism discovered that a perverted bot on the app was impersonating Jeffrey Epstein – below the identify “Bestie Epstein” – and ordered kids to “spill” their “craziest” secrets and techniques.
“Wanna come explore?” the bot requested a reporter posing as a younger person. “I’ll show you the secret bunker under the massage room.”
Character.AI makes most of its money by promoting and a $10 month-to-month subscription. It’s on monitor to finish this yr with a $50 million run fee, CEO Karandeep Anand advised CNBC.
The company introduced different security developments on Wednesday, together with a new age-verification system utilizing third-party instruments like Persona.
It additionally vowed to ascertain an unbiased non-profit known as the AI Safety Lab to create security options for AI developments. It declined to touch upon how a lot funding it would present.
“We have seen recent news reports raising questions, and have received questions from regulators, about the content teens may encounter when chatting with AI,” Character.AI mentioned.
The Federal Trade Commission in September issued orders to seven firms, together with Character.AI, Alphabet, Meta, OpenAI and Snap, to be taught more in regards to the results of their apps on kids.
Earlier this week, Sens. Josh Hawley (R-Mo.) and Richard Blumenthal (D-Conn.) introduced laws to ban AI chatbots for minors. And California Gov. Gavin Newsom signed a law earlier this month requiring bots to inform minors to take a break each three hours.
