No, ChatGPT-4 Can’t Get an MIT Degree – Wccftech
This is not investment advice. The author has no position in any of the stocks mentioned. Wccftech.com has a disclosure and ethics policy.
OpenAI’s ChatGPT is a wonderful tool, albeit flawed in several respects. Leveraging the Large Language Model’s (LLM) capabilities while keeping its limitations in the peripheral vision is the correct approach for now.
Recently, a paper made waves by claiming that ChatGPT-4 can score 100 percent on MIT’s EECS curriculum. What followed, however, is a sordid tale of unethical data sourcing and repeated prompts to obtain the desired outcome. Let’s delve deeper.
🎓 Exploring the MIT Mathematics and EECS curriculum using large language models. 🧠🚀 #AI #Education #MIT Read more: https://t.co/0DxLZj2YhE pic.twitter.com/2YXLrn1RwM
— Iddo Drori (@iddo) June 16, 2023
A few days back, Professor Iddo Drori published a paper titled “Exploring the MIT Mathematics and EECS Curriculum Using Large Language Models.” The paper scrutinized a “comprehensive dataset of 4,550 questions and solutions from problem sets, midterm exams, and final exams across all MIT Mathematics and Electrical Engineering and Computer Science (EECS) courses required for obtaining a degree.” In a striking outcome, the paper concludes:
“Our results demonstrate that GPT-3.5 successfully solves a third of the entire MIT curriculum, while GPT-4, with prompt engineering, achieves a perfect solve rate on a test set excluding questions based on images.”
Given these astonishing claims, the paper went viral on social media, garnering over 500 retweets in a single day.
A recent work from @iddo claimed GPT4 can score 100% on MIT’s EECS curriculum with the right prompting.
My friends and I were excited to read the analysis behind such a feat, but after digging deeper, what we found left us surprised and disappointed.https://t.co/mpDqlenk04
🧵 https://t.co/EVTToZUzL8
— Raunak Chowdhuri (@sauhaarda) June 17, 2023
The paper’s claims were then examined by Raunak Chowdhuri and his colleagues. Contrary to the paper’s assertions, Chowdhuri found glaring problems in the methodology used:
FINAL UPDATE: On June 24th, Armando Solar-Lezama (Professor in EECS and COO/Associate Director of CSAIL, MIT), Tonio Buonassisi (Professor of Mechanical Engineering, MIT), and Yoon Kim (Assistant Professor in EECS and CSAIL, MIT) released a public statement regarding the paper. https://t.co/ITVrK80Hvk pic.twitter.com/BMmlO6GUa6
— Raunak Chowdhuri (@sauhaarda) June 24, 2023
Additionally, a number of MIT professors then issued a statement, disclosing that the paper sourced the MIT dataset without authorization:
“On June 15th, Iddo Drori posted on arXiv a working paper associated with a dataset of exams and assignments from dozens of MIT courses. He did so without the consent of many of his co-authors and despite having been told of problems that should be corrected before publication.”
The statement concludes with the following one-liner:
“And no, GPT-4 cannot get an MIT degree.”
Do you think that ChatGPT’s potential is being damaged by unethical papers? Let us know your thoughts in the comments section below.
Subscribe to get an everyday digest of the latest technology news in your inbox
© 2023 WCCF TECH INC. All rights reserved.
Some posts on wccftech.com may contain affiliate links. We are a participant in the Amazon Services LLC Associates Program, an affiliate advertising program designed to provide a means for sites to earn advertising fees by advertising and linking to amazon.com
© 2023 WCCF TECH INC. All rights reserved.
