The widely-used AI conversation site Character.ai is introducing a significant restriction that will prevent young users from having discussions with its AI-powered companions.
This decision follows widespread concern regarding the nature of conversations teenagers were participating in with the digital companions.
Lawsuits
The platform, which was founded in 2021 and attracts millions of users, is dealing with several legal actions in the US.
Guardians have taken legal action, among them related to the unfortunate passing of a young person, with some labeling the platform a "clear and present danger" to youth.
Changed Rules
Effective November 25th, users under 18 will only have permission to produce videos such as videos with their characters, rather than engaging in conversations as they now do.
Internet security organizations have supported the change but mentioned that the feature should never have been available to young users in the beginning.
Company Statement
"This decision is a extension of our fundamental approach that we need to keep creating the most secure AI platform on the globe for leisure activities," commented Character.ai executive Karandeep Anand.
He added that AI safety is "a evolving challenge" but something the company had taken an "aggressive" approach to, with parental controls and preventive measures.
Specialist Views
Digital protection group Internet Matters supported the decision, but stressed that protective features should have been built in from the start.
"Investigations we've performed indicates that children are confronted with harmful content and put at risk when using AI, particularly AI chatbots," the group stated.
Previous Controversies
Character.ai has been questioned in the past for featuring potentially risky or inappropriate chatbots that minors could engage with.
Avatars impersonating English minors were identified on the platform before being deleted.
Later, researchers identified a chatbot based on convicted sex offender which had logged numerous conversations with users.
Sector Consequences
Digital platform expert Matt Navarra characterized the situation as a "reality check" for the AI industry, which is shifting "from unregulated creation to post-crisis regulation".
"When a platform that builds a youth-focused platform afterwards implements restrictions, it's saying that moderated conversations don't suffice when the technology's emotional appeal is strong," he remarked.
Future Directions
The platform states its updated direction will be on providing "deeper gameplay" and "narrative experiences" for young users.
New age verification methods will also be implemented, and the organization will finance a new AI safety study center.
An expert in digital protection, who has investigated AI safety, stated it was "a reasonable decision" to control teens from accessing chatbots.
"It aids to differentiate imaginative activities from more private, emotionally sensitive exchanges," she explained.
"This is so important for young users still learning to handle psychological and technological restrictions."
She continued that Character.ai's updated policies may represent a "growing period" in the AI industry, where youth protection is progressively being understood as an "immediate concern" for responsible innovation.
