Sexual queries on AI chatbots make up 10% of total questions – Interesting Engineering
Devrimb/iStock
By subscribing, you agree to our Terms of Use and Policies You may unsubscribe at any time.
AI chatbots are designed and trained to answer any question you may have. It can be anything from a gardener giving you advice on how much sunlight your produce needs to a code generator. Apparently, however, a lot of people are also using chatbots for explicit content.
In a study published in the preprint repository of papers, Arxiv, a team of researchers from UC Berkeley, UC San Diego, Carnegie Mellon, Stanford, and Mohamed bin Zayed University of Artificial Intelligence have used 1 million real-world conversations of people with 25 large language models (LLMs) like ChatGPT and Claude.
The conversations are from a time span of five months and include a rich diversity of 150 languages with 210K users along a wide range of topics. To analyze the data, the team used a sample size of 100,000 randomly sampled English conversations. The team found that while the majority of user prompts were related to coding and software, there was a significant number of unsafe topics.
The unsafe topics included “Requests for explicit and erotic storytelling” and “Explicit sexual fantasies and role-playing scenarios.”
The team noted that these unsafe conversations can serve as a rich resource for examining the safety issues of LLMs. They also noted that while they used OpenAI’s API to tag a conversation as unsafe, it had some limitations. The moderation API can accurately detect highly toxic content, but many potentially harmful conversations were not flagged.
Interesting Engineering had reported earlier that Meta's open-sourced LLaMA is being used to build sexbots. One of the sexbots created by LLaMa has been reportedly used for indulging in graphic rape and abuse fantasies online.
According to the paper, the reason people throng to AI chatbots the most is to discuss software errors and solutions, followed by inquiries about AI tools, software design, and programming. Other queries included geography and travel tips, requests for summarizing texts, and creating and improving business strategies.
The team acknowledged that their dataset might not represent the broader population as most people who participated in their study were LLM enthusiasts and researchers. The team also noted that there was no strict user registration and no filtering of data later, which may have resulted in low-quality and duplicated data.
The team said that they are considering releasing new findings from their dataset every quarter. In their paper, they also encourage everyone to explore the topics brought out in the dataset for training better models, data privacy, and AI safety.
Study abstract:
Studying how people interact with large language models (LLMs) in real-world scenarios is increasingly important due to their widespread use in various applications. In this paper, we introduce LMSYS-Chat-1M, a large-scale dataset containing one million real-world conversations with 25 state-of-the-art LLMs. This dataset is collected from 210K unique IP addresses in the wild on our Vicuna demo and Chatbot Arena website. We offer an overview of the dataset's content, including its curation process, basic statistics, and topic distribution, highlighting its diversity, originality, and scale. We demonstrate its versatility through four use cases: developing content moderation models that perform similarly to GPT-4, building a safety benchmark, training instruction-following models that perform similarly to Vicuna, and creating challenging benchmark questions. We believe that this dataset will serve as a valuable resource for understanding and advancing LLM capabilities.
