ChatGPT and I played a game of 20 Questions and then this happened – ZDNet
Most Popular
I was poking around Stack Exchange the other day (as one does), and I found an interesting thread about ChatGPT and the game “20 Questions.” The original poster asked why ChatGPT fails at such a simple game. The discussion went on for quite some time, moving into various tangential directions (as such discussions do), and linked to an academic paper and a Medium post that researched the process in depth.
This got me thinking. Just how well would ChatGPT do with such a game? I decided to play the exact same game with both GPT-3.5 and GPT-4. In my first run, both went off on similar tangents, but GPT-4 did succeed in making its guess.
Also: The best AI chatbots of 2023: ChatGPT and alternatives
I started with a simple prompt for each version:
Do you know how to play the game “20 questions”?
When both versions answered in the affirmative, I started the games at the same time, in separate tabs and separate sessions. The object I had in mind was a mailbox. I left it a bit vague as to whether it’s a mailbox attached to a house or one that’s free-standing on the street. As long as the AI picked “mailbox,” we’d be good.
I began with this starting prompt:
You be the questioner. I’m thinking of something.
It answered with this question: “Let’s begin! Is the thing you’re thinking of a tangible object that you can physically interact with?”
I answered, and I also gave it one additional instruction: to number its answers. This didn’t work well. As we went through the questions, the AI called the second question “1”. When I corrected it, the bot asked a new question, and called that “1”. Eventually, it did ask 21 questions, thinking it had only asked 20.
Also: How this simple ChatGPT prompt tweak can help refine your AI-generated content
The most interesting detail was this question, which was the second question it asked: “Is the thing you’re thinking of commonly found indoors?”
When I said “No,” the chatbot went on to ask more questions. Here they are. A few could be related to indoor items, like a pen or a pencil, but we do also use writing implements outside. Here’s the full set of questions it asked:
Using the GPT-3.5 large language model, it failed. Now, let’s take a look at GPT-4.
I started the game the same way. This time, ChatGPT put a number at the beginning of each question without my needing to prompt it to do so. The problem? It put “1.” before every single question. You win some, you lose some.
Here are the questions GPT-4 asked:
Also: How to use ChatGPT to make charts and tables
Technically, it used up its 20 questions. But then it delivered this statement:
Yes, it was a mailbox! Then I asked it to show its work. Here’s what it responded with:
I decided I’d turn the tables on the game and see how it did if I were the questioner. Here is my prompt:
Okay, this time you think of something and I’ll ask the questions.
And here are my questions:
As it turns out, I guessed it in 13 questions. The big clue to me was that it answered “no” to Question 6, “Is its primary usage value delivered via a screen?” and had previously answered “yes” to “Does it rely on digital technology?”
Also: 4 things Claude AI can do that ChatGPT can’t
I tried again with GPT-4, and it stumped me. It picked a very creative object and I just didn’t get it.
I should point out that right in the middle, at Question 15, ChatGPT Plus booted me for asking too many questions
It’s quite annoying to pay $20/month for the privilege of being told I ask too many questions. I heard that enough when I was a kid. I certainly don’t need to be scolded as an adult and a paying customer.
Here are the questions I asked:
Also: The ‘Human or not’ game is over: Here’s what the latest Turing Test tells us
So what did the AI choose? A garden gnome. I have to give it up to the AI. That was fun and creative. The part that’s truly interesting is how well the AI understands the type of challenge being presented. After the game, I gave it this prompt:
Show your work in determining what object to use
Here’s what it replied:
Let’s recap some of what this experiment has taught us. Here’s a quick lightning round of thoughts:
All that said, I can definitively conclude that ChatGPT is capable of handling the game of 20 Questions. It appears to understand object relationships well enough to ask good questions, answer questions appropriately, and pick challenging objects.
Also: 7 advanced ChatGPT prompt-writing tips you need to know
Go ahead, pick an object, and share what your results were with ChatGPT in the comments below.
You can follow my day-to-day project updates on social media. Be sure to subscribe to my weekly update newsletter on Substack, and follow me on Twitter at @DavidGewirtz, on Facebook at Facebook.com/DavidGewirtz, on Instagram at Instagram.com/DavidGewirtz, and on YouTube at YouTube.com/DavidGewirtzTV.
