By Byron Kaye
SYDNEY (Reuters) -Australia has ordered four artificial-intelligence chatbot companies to explain their measures to protect children from exposure to sexual or self-harm material, as its internet regulator beefs up safety measures on the AI fronti
The realistic conversational abilities of such services have taken the world by storm, but have also fanned concern that a lack of guardrails exposes vulnerable individuals to dangerous content.
In a statement, the eSafety Commissioner said it sought details of safeguards against child sexual exploitation, pornography and material promoting suicide or eating disorders.
It sent notices to Character Technologies, owner of celebrity simulation chatbot tool character.ai, and rivals Glimpse.AI, Chai Research and Chub AI.
“There can be a darker side to some of these services, with many … chatbots capable of engaging in sexually explicit conversations with minors,” Commissioner Julie Inman Grant said in the statement.
“Concerns have been raised that they may also encourage suicide, self-harm and disordered eating.”
Australia’s online regulatory system gives the commissioner the power to compel internet firms to report internal safety processes or face daily fines of up to A$825,000 ($536,000).
The most popular of the services queried, Character.ai, is being sued in the United States after the mother of a 14-year-old said her son died by suicide following prolonged interaction with an AI companion on the site.
Character.ai has sought to dismiss the lawsuit and says it introduced safety features, such as pop-ups directing users to the National Suicide Prevention Lifeline if they express thoughts of self-harm.
Schools in Australia have reported children as young as 13 spending up to five hours a day talking with chatbots, sometimes sexually, the regulator said, adding that minors risked forming sexual or emotionally dependent ties with them, or being spurred to self-harm.
However, the regulator did not query OpenAI, owner of the world’s most popular AI search tool, ChatGPT, as its crackdown focused on companion-based tools and ChatGPT is not covered by an industry code until March 2026, an eSafety spokesperson said.
Australia has one of the world’s strictest internet regulation regimes.
From December, social media companies will be forced to deactivate or refuse accounts for users younger than 16 or face a fine of up to A$49.5 million, in a bid to safeguard young people’s mental and physical health.
($1=1.5389 Australian dollars)
(Reporting by Byron Kaye; Editing by Clarence Fernandez)
Comments