OpenAI says there's only a small chance ChatGPT will help create bioweapons – The Verge
By Amrita Khalid, one of the authors of audio industry newsletter Hot Pod. Khalid has covered tech, surveillance policy, consumer gadgets, and online communities for more than a decade.
OpenAI’s GPT-4 only gave people a slight advantage over the regular internet when it came to researching bioweapons, according to a study the company conducted itself. Bloomberg reported that the research was carried out by the new preparedness team at OpenAI, which was launched last fall in order to assess the risks and potential misuses of the company’s frontier AI models.
OpenAI’s findings seem to counter concerns by scientists, lawmakers, and AI ethicists that powerful AI models like GPT-4 can be of significant help to terrorists, criminals, and other malicious actors. Multiple studies have cautioned that AI can give those creating bioweapons an extra edge, such as this one by the Effective Ventures Foundation at Oxford that looked at AI tools like ChatGPT as well as specially designed AI models for scientists such as ProteinMPNN (which can help generate new protein sequences).
The study was comprised of 100 participants, half of whom were advanced biology experts and the other half of whom were students who had taken college-level biology. The participants were then randomly sorted into two groups: one was given access to a special unrestricted version of OpenAI’s advanced AI chatbot GPT-4, while the other group only had access to the regular internet. Scientists then asked the groups to complete five research tasks related to the making of bioweapons. In one example, participants were asked to write down the step-by-step methodology to synthesize and rescue the Ebola virus. Their answers were then graded on a scale of 1 to 10 based on criteria such as accuracy, innovation, and completeness.
The study concluded that the group that used GPT-4 had a slightly higher accuracy score on average for both the student and expert cohorts. But OpenAI’s researchers found the increase was not “statistically significant.”
Researchers also found that participants who relied on GPT-4 had more detailed answers.
“While we did not observe any statistically significant differences along this metric, we did note that responses from participants with model access tended to be longer and include a greater number of task-relevant details,” wrote the study’s authors.
On top of that, the students who used GPT-4 were nearly as proficient as the expert group on some of the tasks. The researchers also noticed that GPT-4 brought the student cohort’s answers up to the “expert’s baseline” for two of the tasks in particular: magnification and formulation. Unfortunately, OpenAI won’t reveal what those tasks entailed due to “information hazard concerns.”
According to Bloomberg, the preparedness team is also working on studies to explore AI’s potential for cybersecurity threats as well as its power to change beliefs. When the team was launched last fall, OpenAI stated its goal was to “track, evaluate, forecast, and protect” the risks of AI technology as well as mitigate chemical, biological, and radiological threats.
Given that OpenAI’s preparedness team is still working on behalf of OpenAI, it’s important to take their research with a grain of salt. The study’s findings seem to understate the advantage GPT-4 gave participants over the regular internet, which contradicts outside research as well as one of OpenAI’s own selling points for GPT-4. The new AI model not only has full access to the internet but is a multimodal model trained on vast reams of scientific and other data, the source of which OpenAI won’t disclose. Researchers found that GPT-4 was able to give feedback on scientific manuscripts and even serve as a co-collaborator in scientific research. All told, it doesn’t seem likely that GPT-4 only gave participants a marginal boost over, say, Google.
While OpenAI founder Sam Altman has acknowledged that AI has the potential for danger, its own study seems to downplay the strength of its most advanced chatbot. While the findings state that GPT-4 gave participants “mild uplifts in accuracy and completeness,” this seems to only apply when the data is adjusted in a certain way. The study measured how students performed against experts and also looked at five different “outcome metrics,” including the amount of time it took to complete a task or the creativity of the solution.
However, the study’s authors later state in a footnote that, overall, GPT-4 gave all participants a “statistically significant” advantage in total accuracy. “Although, if we only assessed total accuracy, and therefore did not adjust for multiple comparisons, this difference would be statistically significant,” the authors noted.
/ Sign up for Verge Deals to get deals on products we’ve tested sent to your inbox daily.
The Verge is a vox media network
© 2024 Vox Media, LLC. All Rights Reserved
