AI companion chatbot providers must explain how they are keeping kids safe: eSafety – EducationHQ
The eSafety Commissioner has issued legal notices to four well-known AI companion providers requiring them to explain how they are protecting children from exposure to a range of harms, including sexually explicit conversations and images and suicidal ideation and self-harm.
eSafety Commissioner Julie Inman Grant says concerns have been raised that AI companions, powered by generative AI, may encourage suicide, self-harm and disordered eating.
eSafety Commissioner Julie Inman Grant says concerns have been raised that AI companions, powered by generative AI, may encourage suicide, self-harm and disordered eating.
Notices were given to Character Technologies, Inc. (character.ai), Glimpse.AI (Nomi), Chai Research Corp (Chai), and Chub AI Inc. (Chub.ai) under Australia’s Online Safety Act.
The notices require the four companies to answer a series of questions about how they are complying with the Government’s Basic Online Safety Expectations Determination.
The notices require these providers to report on the steps they are taking to keep Australian’s safe online, especially children.
eSafety Commissioner Julie Inman Grant said the AI companions, powered by generative AI, simulate personal relationships through human-like conversations and are often marketed for friendship, emotional support, or even romantic companionship.
“But there can be a darker side to some of these services with many of these chatbots capable of engaging in sexually explicit conversations with minors,” Inman Grant explained.
“Concerns have been raised that they may also encourage suicide, self-harm and disordered eating.
“We are asking them about what measures they have in place to protect children from these very serious harms.
AI companions are increasingly popular, particularly among young people.
One of the most popular, Character.ai, is reported to have nearly 160,000 monthly active users in Australia as of June this year.
“These companies must demonstrate how they are designing their services to prevent harm, not just respond to it. If you fail to protect children or comply with Australian law, we will act,” Inman Grant said.
AI companion providers which fail to comply with a reporting notice could face enforcement action, including court proceedings and financial penalties of up to $825,000 per day.
The notices follow the recent registration of new national industry-drafted codes designed to protect children from exposure to a range of age-inappropriate content that has been occurring at increasingly young ages.
The new codes would also apply to the growing number of AI chatbots that have had limited obligations until recently.
“I do not want Australian children and young people serving as casualties of powerful technologies thrust onto the market without guardrails and without regard for their safety and wellbeing,” Inman Grant said.
The codes and standards are legally enforceable and breach of a direction to comply may result in civil penalties of up to $49.5 million.
Staff
This story was published by the EducationHQ editorial team.
I am really sick of the number of studies that only look at what the outcomes are for the students who are excluded w…
— Christina Bean on Suspending Belief: New Study Reveals Concerning Impacts Of Exclusionary Discipline
In the busy rhythm of the school year, assessment can often feel like a final destination – a measure of what's been taught and learnt.
The eSafety Commissioner has issued legal notices to four well-known AI companion providers requiring them to explain how they are protecting children from exposure to a range of harms, including sexually explicit conversations and images and suic…
The Independent Education Union (IEU) Victoria Tasmania has lodged an historic application at the Fair Work Commission today for a Single Interest Authorisation covering 24 of the 36 employers in Victorian Catholic education.
Maths is seemingly valued more than literacy subjects in NSW’s selective schooling system, and the skewed focus is at the root of a worsening problem, an expert has suggested.
A new report shows that while student uptake of vocational education and training is on the increase, huge discrepancies exist nationwide between jurisdictions and across school sectors, in terms of student numbers and participation levels.
A recent landmark review, supported by Education ministers in all states and territories, recommends a national approach to deal with school bullying and the implementation of trauma-informed training for teachers.
Tasmania’s teachers are poised to take industrial action across the state next week, describing a Government offer of a one-year rollover with a 3 per cent pay rise as ‘insulting’.
It comes as no surprise that what young men see coming across their social media feed is quite different to that of their parents and teachers.
An overwhelming consensus of teachers, parents and students want financial literacy to be taught more in schools, but the reality is it’s not happening in Australia’s crowded curriculum.
Schools will have two days to respond to a bullying complaint or incident under a new $10 million national plan to address the worsening problem – but one approach being pushed is problematic and not grounded in solid evidence, experts have warned.
Australia: +61 3 9229 6200
35-37 Melville Road, Brunswick West VIC 3055 Australia
PO Box 250, Brunswick West VIC 3055 Australia
