Snapchat: Snap AI chatbot 'may risk children's privacy' – BBC.com
Snapchat has been accused of a "worrying failure" to assess the potential privacy risks its AI chatbot poses to users – especially children – by the UK's data watchdog.
The Information Commissioner's Office (ICO) warned it could close down the My AI feature in the UK after a "preliminary investigation".
The US company said it was "closely reviewing" the provisional findings.
The tool lets users message a computer which mimics human conversation.
Snapchat describes it as an "evolving feature" which is powered by ChatGPT, an online AI tool which uses new technology to convincingly imitate realistic responses.
Snap, the parent company behind Snapchat, became the first social media platform to adopt an artificial intelligence-powered chat function earlier this year.
The app has 21 million users in the UK, many of whom are children. The ICO said it was particularly concerned about the potential privacy risks for 13- to 17-year-old users.
Snap said it would "work constructively" with the ICO after it issued a preliminary notice against the company, adding that it had carried out a "robust legal and privacy review" before the function went public.
The data watchdog stressed its findings are not final and it has not concluded that the company breached any data protection laws.
At this stage the notice is a signal to Snap to ensure My AI complies with data protection rules which includes the Children's Design Code.
The watchdog's code contains 15 standards that online services need to follow. This ensures they are complying with their obligations under data protection law to protect children's data online.
The ICO said that if a final enforcement notice was to be adopted, Snap might not be able to offer the My AI function to UK users until the company carries out "an adequate risk assessment".
The company describes My AI as "an experimental and friendly" chatbot designed to be a personal sidekick to each Snapchatter who chats with it.
The feature, which can be used as an assistant to plan day trips or create menus, has more than two million chats per day happening on the app, according to Snap's boss Evan Spiegel.
It was made available to all Snapchat users in April, after being launched for a fee in February.
Since then, the social media platform said "a lot of progress" had been made in its capabilities although it admitted that "mistakes may occur".
"My AI may answer incorrectly, provide biased answers or note it is unsure of the answers so don't rely on its advice," the company said.
Snap has also been criticised for being unclear over whether the chatbot can access private information such as location data.
"Snapchat can only ever access your location if you consent to share it," the firm said.
One of the other concerns about My AI is – because of how young users of Snapchat skew – whether they really understand the implications of data collection.
"Privacy is a foundational value for us – it is critical to our core use case of helping people visually communicate with their friends and family," the platform stressed.
Information Commissioner John Edwards said, "The provisional findings of our investigation suggest a worrying failure by Snap to adequately identify and assess the privacy risks to children and other users before launching My AI.
"We have been clear that organisations must consider the risks associated with AI, alongside the benefits.
"Today's preliminary enforcement notice shows we will take action in order to protect UK consumers' privacy rights."
In the case of serious breaches, the ICO has the power to issue fines of £17.5 million or 4% of a company's annual worldwide turnover from the preceding financial year, whichever is higher.
A Snap spokeswoman said: "We are closely reviewing the ICO's provisional decision.
"Like the ICO, we are committed to protecting the privacy of our users.
"In line with our standard approach to product development, My AI went through a robust legal and privacy review process before being made publicly available.
"We will continue to work constructively with the ICO to ensure they're comfortable with our risk assessment procedures."
Snap will now have a chance to respond to the regulator's concerns before the ICO makes its final decision.
Snapchat introduces AI chatbot to mixed reviews
Hundreds of foreign nationals and injured leave Gaza as border finally opens
Donald Trump Jr testifies in NY civil fraud trial
Relaxed Trump Jr cracks jokes in courtroom debut
Children must be off limits, says father of abducted kids
'I don’t want to die at 24' – trapped Gaza woman
Weyes Blood: I feel like I've lived a million lives
Why Trump's children are testifying in fraud case
Roses, cars and jeans: Has US trade deal helped Africa?
Why are fewer women using AI than men?
Gun battles and car crashes plague Europe migrant route
Political violence grips Bangladesh as election looms
The children forced from home by Russian attacks
The dark side of touch-screen tipping
Europe's magnificent €50 train
The British obsession with footballers' wives
© 2023 BBC. The BBC is not responsible for the content of external sites. Read about our approach to external linking.
