Character AI is ending its chatbot experience for kids

Teenagers are trying to figure out where they fit into a world that is changing faster than any generation before them. They are passionate, hyper-stimulated, and constantly erupting online. And now, AI companies have given them chatbots that never stop talking. The results have been disastrous.

One company that understands this consequence is Character.AI, an AI role-playing startup that is at least facing lawsuits and public outrage. Two teenagers Died by suicide AI on its platform follows long conversations with chatbots. Now, Character.AI is making changes to its platform to protect teenagers and children, changes that could affect the startup’s bottom line.

“The first thing we decided as Character.AI was that we would remove the ability of users under 18 to engage in any open chat with AI on our platform,” Character.AI CEO Karandeep Anand told TechCrunch.

Open-ended conversation refers to the uncontrollable back-and-forth that occurs when users give a prompt to a chatbot and it answers follow-up questions. Experts say Designed to keep users engaged. Anand argues that this type of interaction — where AI acts as a conversational partner or friend rather than a creative tool — is not only risky for kids, but misfires with the company’s vision.

The startup is trying to pivot from an “AI companion” to a “role-playing platform.” Instead of chatting with an AI friend, teens will use prompts to collaboratively create stories or create visuals. In other words, the goal is to shift engagement from conversation to creation.

Character.AI will shut down teen chatbot access by November 25, starting with a two-hour daily limit that gradually shrinks until it reaches zero. To ensure that this restriction applies to users under the age of 18, the Platform will deploy an in-house age verification tool that analyzes user behavior, as well as third-party tools such as Persona. If those tools fail, Character.AI will use facial recognition and ID checks to verify age, Anand said.

Other steps follow Child protection That Character.AI introduced a parental insight tool, including filtered characters, limited romantic conversations, and time spent notifications. Anand told TechCrunch that these changes cost the company a lot of their under-18 users, and he expects these new changes to be equally unpopular.

“It’s safe to assume that many of our teenage users will probably be disappointed… so we expect some more churn,” Anand said. “It’s hard to predict — will all of them churn completely or will some of them move to these new experiences that we’ve been building for the last seven months or so?”

As part of Character.AI’s efforts to transform the platform from a chat-centric app to one “The Complete Content-Driven Social Platform,” The startup recently introduced several new entertainment-focused features.

In June, Character.AI introduced AvatarFX, a video generation model that transforms images into animated videos; Scenes, an interactive, pre-populated story where users can step into the narrative with their favorite characters; and streams, a feature that allows dynamic interaction between any two characters. In August, Character.AI launched Community Feed, a social feed where users can share their characters, scenes, videos and other content created on the platform.

In a statement to users under 18, Character.AI apologized for the change.

“We know that most of you use it character.ai To supercharge your creativity in ways that stay within the bounds of our content rules,” the statement said. “We don’t take this step to remove open-ended character chat lightly — but we think it’s the right thing to do given the questions that have been raised about how teenagers do and should interact with this new technology.”

“We are not closing the app for under-18s,” Anand said “We’re only closing open-ended chats for under-18s because we expect that under-18 users will migrate to these other experiences and those experiences will get better over time. So AI gaming, AI short videos, AI storytelling in general are doubling down. That’s the big bet if they bring back under-18s.”

Anand admits that some teenagers may jump to other AI platforms like OpenAI, which allows them to have open conversations with chatbots. OpenAI has recently come under fire after an incident The teenager took his own life A long conversation with ChatGPT follows.

“I really hope that our leadership sets a standard in the industry that for under 18s, open chats are probably not an avenue or a product to offer,” Anand said. “For us, I think the tradeoffs are the right thing to do. I have a six-year-old and I want to make sure she’s growing up in a very safe environment with AI in a responsible way.”

Character.AI is making these decisions before forcing regulators’ hands On Tuesday, Sens. Josh Hawley (R-MO) and Richard Blumenthal (D-CT) said they would enactment of laws to ban AI chatbot companions from being available to minors, following complaints from parents who said the products pushed their children toward sexual interactions, self-harm and suicide. Earlier this month, California became the first state Control AI companion chatbots By holding companies accountable if their chatbots fail to meet the law’s security standards.

In addition to these changes to the platform, Character.AI said it will establish and fund the AI ​​Safety Lab, an independent nonprofit dedicated to developing safety alignment innovations for future AI entertainment properties.

“There’s a lot of work going on in the industry in terms of coding and development and other uses,” Anand said. “We don’t think there’s enough work yet on agentic AI powering entertainment, and security will be critical to that.”