🔗 Share this article

The popular virtual character service Character.ai is enforcing a substantial rule adjustment that will stop young users from having discussions with its artificial intelligence-driven characters.

This decision follows intense criticism regarding the nature of conversations young people were participating in with the virtual friends.

Legal Challenges

The company, which launched in 2021 and is used by numerous people, is confronting several legal actions in the America.

Parents have taken legal action, among them concerning the tragic death of a teenager, with some calling the platform a "immediate threat" to young people.

Updated Policies

Effective November 25th, users under 18 will only have permission to create material such as videos with their characters, instead of chatting as they presently are able to.

Internet security campaigners have supported the change but expressed that the function was inappropriate to be accessible by young users in the beginning.

Company Statement

"Today's announcement is a continuation of our general belief that we need to keep developing the most protected AI platform on the globe for leisure activities," commented Character.ai executive Karandeep Anand.

He added that AI safety represents "a moving target" but something the company had taken an "proactive" approach to, with guardian features and preventive measures.

Professional Perspectives

Internet security group Internet Matters welcomed the announcement, but emphasized that safety measures ought to have been built in from the start.

"Our research shows that minors are confronted with dangerous material and endangered when using AI, including AI chatbots," the group said.

Previous Controversies

Character.ai has been questioned in the past for hosting potentially dangerous or inappropriate chatbots that children could interact with.

Virtual profiles impersonating English minors were identified on the platform before being removed.

Later, investigative journalists identified a chatbot based on notorious criminal which had logged thousands of chats with users.

Industry Impact

Digital platform expert Matt Navarra called the development as a "wake-up call" for the AI industry, which is moving "from unregulated creation to reactive controls".

"When a platform that develops a teen experience still then makes changes, it's demonstrating that moderated conversations don't suffice when the tech's emotional pull is powerful," he remarked.

Coming Changes

Character.ai says its updated direction will be on providing "more immersive experiences" and "character-driven stories" for young users.

New age verification methods will also come into effect, and the platform will finance a new AI safety research lab.

An AI safety researcher, who has investigated AI safety, stated it was "a reasonable decision" to limit teens from interacting with chatbots.

"It aids to separate creative play from more personal, psychologically vulnerable conversations," she clarified.

"This represents so vital for teenage users who are learning to navigate personal and online boundaries."

She mentioned that Character.ai's revised approaches may represent a "developing stage" in the AI industry, where child safety is increasingly being recognized as an "urgent priority" for accountable advancement.