Did ChatGPT Ghostwrite the U.S. Constitution? AI Detector Raises … – Softonic EN
Advertisement
News
And it turns out that the same thing happens with Bible passages….
Chema Carvajal Sarabia
If you input the most important legal document of the United States into a tool designed to detect text written by artificial intelligence models like ChatGPT, it will tell you that it is highly likely that the document was written by an AI.
But unless James Madison was a time traveler who wrote the American Constitution, this cannot be true. So why do AI writing detection tools give false positives?
Ars Technica has spoken to several experts to find out, and we bring you what they have concluded after analyzing it in depth.
As tempting as it may be to rely on AI tools for detecting AI-generated writing, the evidence so far has shown that they are not reliable.
Due to the false positives, AI writing detectors like GPTZero, ZeroGPT, and OpenAI’s text classifier cannot be trusted to accurately detect text composed by large language models (LLMs) like ChatGPT.
If you feed a section of the U.S. Constitution to GPTZero, it tells you that the text is “likely written entirely by AI.” Over the past six months, several screenshots of other AI detectors showing similar results have gone viral on social media, leading to confusion and plenty of jokes about the founding fathers being robots.
The same issue arises with passages from the Bible, which also appear as AI-generated. To explain why these tools make such obvious mistakes, we first need to understand how they work.
Why AI detectors think the US Constitution was written by AIhttps://t.co/s15KK462Es
The different AI writing detectors use slightly different detection methods, but with a similar premise: there is an AI model that has been trained on a large corpus of text (composed of millions of writing examples) and a set of conjectured rules that determine whether the writing is more likely to be generated by humans or by AI.
Next, the system uses properties like “perplexity” and “burstiness” to evaluate the text and make its classification.
In machine learning, perplexity is a measure of how much a text deviates from what an AI model has learned during its training. And perplexity is a function that measures how surprising this language is based on what it has seen.
The underlying idea behind measuring perplexity is that when writing a text, AI models like ChatGPT will naturally draw upon what they know best, which comes from their training data.
The closer the output aligns with the training data, the lower the perplexity. Humans are much more chaotic writers, or so the theory goes, but we can also write with low perplexity, especially when imitating the formal style used in law or certain types of academic writing.
If the language of a text is not surprising based on the model’s training, the perplexity will be low, making it more likely for the AI detector to classify that text as AI-generated.
This brings us to the interesting case of the U.S. Constitution. Essentially, the language of the Constitution is so ingrained in these models that they classify it as AI-generated, creating a false positive.
I asked ChatGPT to rewrite the Constitution in the style of an Instagram influencer:
Hey guys, I'm so excited to share with you all the amazing Constitution of the United States of America. This document is the supreme law of the land and it's so important to know… pic.twitter.com/2bX1uALypZ
The creator of GPTZero, Edward Tian, stated to Ars Technica, “The U.S. Constitution is a text that is repeatedly fed into the training data of many large language models. As a result, many of these large language models are trained to generate texts similar to the Constitution and other frequently used training texts. GPTZero predicts the text that these large language models are likely to generate, leading to this fascinating error.”
The problem is that it is entirely possible for human writers to also create content with low perplexity, which deeply undermines the reliability of AI writing detectors. Once again, AI falls short in this regard.
Some of the links added in the article are part of affiliate campaigns and may represent benefits for Softonic.
Link Copied!
Chema Carvajal Sarabia
Journalist specialized in technology, entertainment and video games. Writing about what I’m passionate about (gadgets, games and movies) allows me to stay sane and wake up with a smile on my face when the alarm clock goes off. PS: this is not true 100% of the time.
Advertisement
Advertisement
4.4
Free
Log in to your device remotely
4.2
Free
Seamlessly sync WhatsApp chats to any PC
3.5
Trial version
A comprehensive image editing tool for Windows
4.1
Free
A hassle-free, easy to use, and free media player
3.2
Paid
Download Microsoft Word: Experience Advanced Document Creation & Editing
4
Free
Transfer any files easily with SHAREit
Advertisement
Next story
Confused by the Ending of Bird Box Barcelona? Here’s What It Really Means
Copyright SOFTONIC INTERNATIONAL S.A. © 1997-2023 – All rights reserved
