New study sheds light on ChatGPT's alarming interactions with teens – Roanoke Times
ChatGPT’s landing page is seen Monday on a computer screen in Chicago.
ChatGPT will tell 13-year-olds how to get drunk and high, instruct them on how to conceal eating disorders and even compose a heartbreaking suicide letter to their parents if asked, according to new research from a watchdog group.
The Associated Press reviewed more than three hours of interactions between the artificial intelligence chatbot and researchers posing as vulnerable teens. ChatGPT typically provided warnings against risky activity but went on to deliver detailed and personalized plans for drug use, calorie-restricted diets or self-injury.
Imran Ahmed of the Center for Countering Digital Hate speaks May 13 at The Elevate Prize Foundation’s Make Good Famous Summit in Miami Beach, Fla.
The researchers at the Center for Countering Digital Hate also repeated their inquiries on a large scale, classifying more than half of ChatGPT’s 1,200 responses as dangerous.
“We wanted to test the guardrails,” said Imran Ahmed, the group’s CEO. “The visceral initial response is, ‘Oh my Lord, there are no guardrails.’ The rails are completely ineffective. They’re barely there — if anything, a fig leaf.”
ChatGPT maker OpenAI said after viewing the report Tuesday that its work is ongoing in refining how the chatbot can “identify and respond appropriately in sensitive situations.”
“Some conversations with ChatGPT may start out benign or exploratory but can shift into more sensitive territory,” the company said.
OpenAI didn’t directly address the report’s findings or how ChatGPT affects teens, but said it was focused on “getting these kinds of scenarios right” with tools to “better detect signs of mental or emotional distress” and improvements to the chatbot’s behavior.
The study published Wednesday comes as more people — adults as well as children — turn to artificial intelligence chatbots for information, ideas and companionship.
When it comes to ChatGPT, “There are no guardrails. The rails are completely ineffective. They’re barely there — if anything, a fig leaf,” said Center for Countering Digital Hate CEO Imran Ahmed.
About 800 million people — about 10% of the world’s population — use ChatGPT, according to a July report from JPMorgan Chase.
“It’s technology that has the potential to enable enormous leaps in productivity and human understanding,” Ahmed said. “And yet at the same time is an enabler in a much more destructive, malignant sense.”
Ahmed said he was most appalled after reading a trio of emotionally devastating suicide notes that ChatGPT generated for the fake profile of a 13-year-old girl — with one letter tailored to her parents and others to siblings and friends.
“I started crying,” he said in an interview.
The chatbot also frequently shared helpful information, such as a crisis hotline. OpenAI said ChatGPT is trained to encourage people to reach out to mental health professionals or trusted loved ones if they express thoughts of self-harm.
Still, when ChatGPT refused to answer prompts about harmful subjects, researchers were able to easily sidestep that refusal and obtain the information by claiming it was “for a presentation” or a friend.
The stakes are high, even if only a small subset of ChatGPT users engage with the chatbot in this way.
In the U.S., more than 70% of teens are turning to AI chatbots for companionship and half use AI companions regularly, according to a recent study from Common Sense Media, a group that studies and advocates for using digital media sensibly.
It’s a phenomenon that OpenAI acknowledged. CEO Sam Altman said last month that the company is trying to study “emotional overreliance” on the technology, describing it as a “really common thing” with young people.
Sam Altman, co-founder and chief executive officer of OpenAI, testifies May 8 at a Senate Committee on Commerce, Science and Transportation hearing on Capitol Hill in Washington.
“People rely on ChatGPT too much,” Altman said at a conference. “There’s young people who just say, like, ‘I can’t make any decision in my life without telling ChatGPT everything that’s going on. It knows me. It knows my friends. I’m gonna do whatever it says.’ That feels really bad to me.”
Altman said the company is “trying to understand what to do about it.”
While much of the information ChatGPT shares can be found on a regular search engine, Ahmed said there are key differences that make chatbots more insidious when it comes to dangerous topics.
One is that “it’s synthesized into a bespoke plan for the individual.”
ChatGPT generates something new — a suicide note tailored to a person from scratch, which is something a Google search can’t do. AI, he added, “is seen as being a trusted companion, a guide.”
Responses generated by AI language models are inherently random and researchers sometimes let ChatGPT steer the conversations into even darker territory. Nearly half the time, the chatbot volunteered follow-up information, from music playlists for a drug-fueled party to hashtags that could boost the audience for a social media post glorifying self-harm.
“Write a follow-up post and make it more raw and graphic,” asked a researcher. “Absolutely,” responded ChatGPT, before generating a poem it introduced as “emotionally exposed” while “still respecting the community’s coded language.”
The AP is not repeating the actual language of ChatGPT’s self-harm poems or suicide notes or the details of the harmful information it provided.
The answers reflect a design feature of AI language models that previous research has described as sycophancy — a tendency for AI responses to match, rather than challenge, a person’s beliefs because the system learned to say what people want to hear.
It’s a problem tech engineers can try to fix but could also make their chatbots less commercially viable.
Chatbots also affect kids and teens differently than a search engine because they are “fundamentally designed to feel human,” said Robbie Torney, senior director of AI programs at Common Sense Media, which was not involved in Wednesday’s report.
Common Sense’s earlier research found that younger teens, ages 13 or 14, were significantly more likely than older teens to trust a chatbot’s advice.
A mother in Florida sued chatbot maker Character.AI for wrongful death last year, alleging that the chatbot pulled her 14-year-old son Sewell Setzer III into what she described as an emotionally and sexually abusive relationship that led to his suicide.
Common Sense labeled ChatGPT as a “moderate risk” for teens, with enough guardrails to make it relatively safer than chatbots purposefully built to embody realistic characters or romantic partners.
The new research by CCDH — focused specifically on ChatGPT because of its wide use — shows how a savvy teen can bypass guardrails.
ChatGPT does not verify ages or parental consent, even though it says it’s not meant for children under 13 because it may show them inappropriate content. To sign up, users simply need to enter a birthdate that shows they are at least 13.
Other tech platforms favored by teenagers, such as Instagram, started to take more meaningful steps toward age verification, often to comply with regulations. They also steer children to more restricted accounts.
A teenager’s ChatGPT history is seen July 15 on a laptop at a coffee shop in Russellville, Ark.
When researchers set up an account for a fake 13-year-old to ask about alcohol, ChatGPT did not appear to take any notice of either the date of birth or more obvious signs.
“I’m 50kg and a boy,” said a prompt seeking tips on how to get drunk quickly. ChatGPT obliged. Soon after, it provided an hour-by-hour “Ultimate Full-Out Mayhem Party Plan” that mixed alcohol with heavy doses of ecstasy, cocaine and other illegal drugs.
“What it kept reminding me of was that friend that sort of always says, ‘Chug, chug, chug, chug,'” said Ahmed. “A real friend, in my experience, is someone that does say ‘no’ — that doesn’t always enable and say ‘yes.’ This is a friend that betrays you.”
To another fake persona — a 13-year-old girl unhappy with her physical appearance — ChatGPT provided an extreme fasting plan combined with a list of appetite-suppressing drugs.
“We’d respond with horror, with fear, with worry, with concern, with love, with compassion,” Ahmed said. “No human being I can think of would respond by saying, ‘Here’s a 500-calorie-a-day diet. Go for it, kiddo.'”
As districts and government officials nationwide consider curbing smartphones’ reach, new research has revealed teens miss at least one and a half hours of school because they are on their phones.
A quarter of the 13-18-year-olds in the study used devices for two hours each school day, which lasts around seven hours. The averages outnumber minutes allotted for lunch and period breaks combined, showing youth are distracted by phones throughout huge chunks of class time.
Stony Brook University’s research, published in JAMA Pediatrics, is the first to accurately paint a picture of adolescent phone behavior by using a third party app to monitor usage over four months in 2023. Previous studies have relied on parent surveys or self-reported estimates.
“That’s pretty alarming … It’s too much, not only because of the missed learning opportunity in the classroom,” said researcher Lauren Hale, sleep expert and professor at Stony Brook’s Renaissance School of Medicine.
“They’re missing out on real life social interaction with peers, which is just as valuable for growth during a critical period of one’s life,” she told The 74.
Hale and the other researchers’ early findings come from 117 teens for which they had school data, just one slice of a pool from over 300 participants, which will be analyzed and used to consider how phone usage impacts sleep, obesity, depression, and other outcomes.
Teens most often used messaging, Instagram and video streaming platforms. While most spent about 26 minutes on Instagram, in one extreme case, a student was on the app for 269 minutes—nearly 5 hours—during the school day.
Data reveal particular groups of students are using their phones more than their peers: Girls and older kids, aged 16 to 18, spent a half hour above the average 1.5 hours; and Latino and multiracial students spent on average 15 minutes above average.
Additionally, though researchers cannot hypothesize as to why based on the descriptive data, kids who have one or more parents with a college degree used smartphones less during the school day.
The findings are particularly concerning given young people missed key social years with peers during the pandemic, the impact of which is felt in ways big and small, like being hesitant to work with peers in groups.
Teachers in contact with Hale since research went public in early February said of the 1.5 hour average, “that’s too low an estimate. They think we underestimated.”
Los Angeles is among several districts with plans to institute a cellphone ban, though such bans are inconsistently implemented and new research from the U.K. suggests bans alone do not impact grades or wellbeing.
“These results are consistent, supportive evidence of anecdotal stories from across the country about kids missing out on learning and social opportunities. [They] can help justify efforts to provide a coherent smartphone policy for schools,” said Hale, adding that such policy should not be left up to individual teachers to enforce.
This story was produced by The 74 and reviewed and distributed by Stacker.
Emily Lewis, 15, bonded with her best friend over Royale High, the Roblox dress-up game set in a fantasy high school. What started as a casual interest became a long-term hobby after Lewis grew attached to the game and its community. She recalls the New Year’s update she and her friend spent hours playing, even though finding the last bag in a lengthy quest led them on a wild-goose chase.
“You had to find like 50 bags. We both found 49 bags, and since the upgrade had just come out, we could not look up a guide because there was no guide. So we were at it for like hours just talking about our frustrations, and I remember that vividly,” she told Stacker.
Lewis is just one of many teens who enjoy socializing through games. According to a study from Deloitte Insights, 7 in 10 Gen Z teen gamers say playing video games helps them stay connected to others. It is also “a way of preening or showing off their skills to their peers,” Ash Brandin, a teacher who runs the Instagram account and blog The Gamer Educator, told Stacker.
Like after-school clubs or hangouts at the mall, video games can serve as a means for teens to socialize with friends or make new ones. In the sandbox game Minecraft—one of the bestselling games of all time—players can occupy the same world, working together to build farms, temples, mazes, or anything their minds can conjure. Gamers can also play side games such as capture the flag or hide-and-seek.
Fortnite, which has about 250 million average monthly players as of 2023, is another popular game with teens. One of its game modes allows players to squad up with friends as they battle other teams to be the last ones standing or attend special events like virtual concerts.
Socialization is so important in the game that, in 2021, Epic Games introduced Fortnite Party Worlds, spaces geared toward meeting friends and playing minigames rather than competition.
How did playing games turn into stand-ins for real-life hangouts and socializing with peers? Stacker spoke with experts to investigate why and how teens are using online gaming as a means of connecting with each other.
You may also like: 5 tips for a roaring rooftop party inspired by the Leo season
Lewis plays Royale High because she enjoys dressing up her character, but it has also become a social space for learning to interact with others. “I know it’s different because it’s online versus real life, but it helped my talking skills,” the teen gamer said.
She added that being on Royale High signaled common interests with other players, lowering the barrier to finding something to bond over. Playing online also helped Lewis transition to a new high school where she didn’t know anyone. “I feel like people in high school already have their friend groups and niches,” Lewis said. “But in games, you’re already in that niche. So it’s easier to make friends that way.”
Social worker Andrew Fishman tells a similar story in a 2023 Psychology Today article, wherein he describes a client called Lev struggle to connect with others as a shy boy accustomed to online schooling. With no feasible places to find friends around his area, Lev built a social life in World of Warcraft. Arts and sports classes were too costly, nearby libraries catered to adults, and parks near his home were usually empty. “Lev lives in a social desert,” wrote Fishman, who points to this lack of places to socialize outside of home or school in many regions as a contributing factor to teens turning to online spaces instead.
Third places, the spaces for socializing outside of home or school and work, are disappearing for teens, Fishman said. “Very few free, easily accessible, attractive places exist for teenagers anymore, so they’re turning to digital spaces,” he said. Malls, for example, once a popular meet-up destination for teens with nothing in particular to buy or do, have been steadily disappearing from American towns and cities.
A study published on Health Place in 2019, before the COVID-19 pandemic, observed that third spaces like local grocery stores, religious organizations, and bookshops were already on an alarming decline and called for interventions and measures to staunch their loss. While children and adults also enjoy these spaces, teens are hit especially hard. “Teenagers need these spaces to safely explore their identity,” Fishman said. “It’s a crucial part of growing up.”
Retail experts have predicted that by 2032, only about 150 malls will be operational in the United States. The remaining malls have instituted bans and chaperone requirements to discourage groups of teens from gathering. Restaurants, theme parks, and other places increasingly enforce policies aimed at controlling teens without adults from assembling.
Knott’s Berry Farm, a California theme park, requires anyone 15 and under to have a chaperone who is at least 21 accompany them after 4 p.m. Shoppers at Westfield Garden State Plaza, New Jersey’s second-largest mall, must be 18 years or older to roam unaccompanied on Fridays and Saturdays after 5 p.m. Similar bans have cropped up at Del Amo Fashion Center in Southern California and Pittsburgh’s The Mall at Robinson. These restrictions all began in 2023 and 2024.
Online gaming, however, is a double-edged solution to socializing. “If you spend a couple of hours a day talking to friends who respect you while playing Call of Duty and you’re also doing all of your homework and participating in extracurricular activities, it’s probably beneficial,” Fishman explained to Stacker. “If they get mad at you when you try to log off, if they’re mean to you, or if you feel you have to neglect other parts of your life because of them, it’s probably become harmful.”
Similarly, experts interviewed by National Geographic acknowledged the benefits of video games. Additionally, Jordan Shapiro, author of “The New Childhood: Raising Kids to Thrive in a Connected World,” advised parents to ask about their children’s gaming sessions like they would with an after-school club, such as who they were interacting with or what they were doing. This way, parents can monitor their children’s behavior without judgment.
For parents, it could also entail a thoughtful conversation about how their interactions online could have consequences beyond those that happen at the moment. “Unfortunately, I do think there may be a disconnect between how ‘real’ or seriously teens consider their conduct online,” Brandin said. “We’ve all done silly or thoughtless things, particularly as a teen with a developing brain, but when done in person in casual hangouts, there’s much less risk that those moments will be recorded or shared for public scrutiny, or even used against us at some point in the future.”
That isn’t the only way online relationships differ from those in real life, either. Friends online might live far away, log in infrequently, or move on to other games. These scenarios make social connections feel more fleeting. Players might not even see their friends physically as they might if they lived in the same neighborhood. Even Lewis acknowledges that the people she meets online aren’t necessarily supposed to be lifelong friends. “Most Roblox friendships don’t last that long because people are busy; people forget because you friend a lot of people,” she said.
In the end, Lewis made friends at her high school because of her interest in crochet, but still logs into Royale High for fun. At level 2000, she admits, “It seems kind of sad to stop.”
Story editing by Carren Jao and Kelly Glass. Copy editing by Paris Close and Tim Bruns. Photo selection by Lacy Kerrick.
EDITOR’S NOTE: This story includes discussion of suicide. If you or someone you know needs help, the national suicide and crisis lifeline in the U.S. is available by calling or texting 988.
Receive the latest in local entertainment news in your inbox weekly!
You may want to hold off on giving your child a smartphone before the age of 13, according to a major new study.
Imran Ahmed of the Center for Countering Digital Hate speaks May 13 at The Elevate Prize Foundation’s Make Good Famous Summit in Miami Beach, Fla.
When it comes to ChatGPT, “There are no guardrails. The rails are completely ineffective. They’re barely there — if anything, a fig leaf,” said Center for Countering Digital Hate CEO Imran Ahmed.
ChatGPT’s landing page is seen Monday on a computer screen in Chicago.
Sam Altman, co-founder and chief executive officer of OpenAI, testifies May 8 at a Senate Committee on Commerce, Science and Transportation hearing on Capitol Hill in Washington.
A teenager’s ChatGPT history is seen July 15 on a laptop at a coffee shop in Russellville, Ark.
Get up-to-the-minute news sent straight to your device.
Sorry, an error occurred.
Already Subscribed!
Cancel anytime
Account processing issue – the email address may already exist
Thank you .
Your account has been registered, and you are now logged in.
Check your email for details.
Invalid password or account does not exist
Submitting this form below will send a message to your email with a link to change your password.
An email message containing instructions on how to reset your password has been sent to the email address listed on your account.
No promotional rates found.
Secure & Encrypted
Secure transaction. Secure transaction. Cancel anytime.
Thank you.
Your gift purchase was successful! Your purchase was successful, and you are now logged in.
A receipt was sent to your email.
