Why Is ChatGPT-4 So Slow Compared to ChatGPT-3.5? – MUO – MakeUseOf
It’s newer but slower. What gives?
With the newest version of ChatGPT, GPT-4, released in March 2023, many are now wondering why it is so slow compared to its predecessor, GPT-3.5. So, what's the core reason here?
Just why is ChatGPT-4 so sluggish, and should you stick to GPT-3.5 instead?
ChatGPT-4 is the newest model of OpenAI's chatbot, known generally as ChatGPT. ChatGPT is powered by artificial intelligence, allowing it to answer your questions and prompts far better than previous chatbots. ChatGPT uses a large language model powered by a GPT (Generative Pre-trained Transformer) to provide information and content to users while also being able to converse.
ChatGPT has a wide range of capabilities, making it useful for millions. For example, ChatGPT can write stories, formulate jokes, translate text, educate users, and more. While ChatGPT can also be used for more illicit acts, such as malware creation, its versatility is somewhat revolutionary.
ChatGPT's GPT-4 model was released on March 14, 2023. This version of ChatGPT is designed to better understand emotional language through text while also better understanding different language dialects and image processing. GPT-4 can also hold longer conversations and effectively respond to longer user prompts.
Additionally, GPT-4's parameters exceed those of GPT-3.5 by a large extent. ChatGPT's parameters determine how the AI processes and responds to information. In short, parameters determine the skill the chatbot has to interact with users. While GPT-3.5 has 175 billion parameters, GPT-4 has an incredible 100 trillion to 170 trillion (rumored—OpenAI hasn't confirmed this figure).
It was OpenAI's GPT-3.5 that was used to power ChatGPT, which is now the most popular AI chatbot in the world. So, GPT-3.5 has made an undeniable mark in the AI realm. But things are always progressing in the tech industry, so it's no surprise that GPT-3.5 now has a successor in GPT-4.
However, GPT-4 is by no means perfect. In fact, GPT-4's lengthy response times are causing quite a stir. So let's look into this issue and why it may be happening.
Many noticed upon the release of GPT-4 that OpenAI's new chatbot was incredibly slow. This left scores of users frustrated, as GPT-4 was meant to be a step up from GPT-3.5, not backward. As a result, GPT-4 users have been taking to online platforms, such as Reddit and OpenAI's community board, to discuss the issue.
On OpenAI's Community Forum, a number of users have come forward with their GPT-4 delay frustrations. One user stated that GPT-4 was "extremely slow" on their end and that even small requests made to the chatbot resulted in unusually long delays of over 30 seconds.
Other users were quick to share their experiences with GPT-4, with one commenting under the post that "the same call with the same data can take up to 4 times slower than 3.5 turbo."
In another OpenAI Community Forum post, a user commented that their prompts are sometimes met with an "error in body stream" message, resulting in no response. In the same thread, another individual stated they couldn't get GPT-4 to "successfully respond with a complete script." Another user commented that they kept running into network errors while trying to use GPT-4.
With delays and failed or half-baked responses, it seems that GPT-4 is littered with issues that are quickly putting users off.
So why, exactly, is this happening? Is there something wrong with GPT-4?
In the OpenAI Community Forum post referenced above, one user responded suggesting that the delay was due to a "current problem with whole infrastructure overload," adding that there is a challenge posed in "tackling scalability in such a short time frame with this popularity and number of users of both chat and API."
In a Reddit post uploaded in the r/singularity subreddit, a user laid out a few possible reasons for GPT-4's slowness, starting with a larger context size. Within the GPT ecosystem, context size refers to how much information a given chatbot version can process and then produce information. While GPT-3.5's context size was 4K, GPT-4's is double that. So, having an 8K context size may be having an impact on GPT-4's overall speeds.
The Reddit author also suggested that the enhanced steerability and control of GPT-4 could play a role in the chatbot's processing times. Here, the author stated that GPT-4's greater steerability and control of hallucinations and inappropriate language might be the culprits, as these features add extra steps to GPT-4's method of processing information.
Furthermore, it was proposed that GPT-4's ability to process pictures could be slowing things down. This useful feature is loved by many but could come with a catch. Given that it has been rumored GPT-4 takes 10-20 seconds to process a provided image, there's a chance that this component is stretching out response times (though this doesn't explain the delays experienced by users providing text prompts only).
Other users have suggested that the newness of ChatGPT-4 is playing a big role in these delays. In other words, some think that OpenAI's newest chatbot needs to experience some growing pains before all flaws can be ironed out.
But the biggest reason GPT-4 is slow is the number of parameters GPT-4 can call upon versus GPT-3.5. The phenomenal rise in parameters simply means it takes the newer GPT model longer to process information and respond accurately. You get better answers with increased complexity, but getting there takes a little longer.
So, with these issues in mind, should you use GPT-3.5 or GPT-4?
At the time of writing, it seems GPT-3.5 is the snappier option over GPT-4. So many users have experienced delays that it's likely the time issue is present across the board, not just with a few individuals. So, if ChatGPT-3.5 is currently meeting all your expectations, and you don't want to wait around for a response in exchange for extra features, it may be wise to stick to this version for now.
However, you should note that GPT-4 isn't just GPT-3.5 but slower. This version of OpenAI's chatbot has numerous advantages over its predecessor. If you're looking for a more advanced AI chatbot and don't mind waiting longer for responses, it may be worth transitioning from GPT-3.5 to GPT-4.
Over time, GPT-4's delays may be lessened or entirely resolved, so patience could be a virtue here. Whether you try switching to GPT-4 now or wait a little longer to see how things play out with this version, you can still get a lot out of OpenAI's nifty little chatbot.
While GPT-4 has numerous advanced capabilities over GPT-3.5, its significant delays and response errors have made it unusable to some. These issues may be resolved in the near future, but for now, GPT-4 certainly has some obstacles to overcome before it is accepted on a wider scale.
Katie has been writing about tech-based topics for two years, with a specific interest in cybersecurity, AI, and cryptocurrency. Katie has covered a variety of topics during her time at MUO, including crypto explainers, cybersecurity guides, VPN reviews, recent hacks, and software tutorials. With a passion for emerging tech, Katie is also excited to see what new devices and digital platforms the coming years will bring.
