Complexity a nemesis of AI: ChatGPT stumped by nuanced health queries, report shows – McKnight's Senior Living
ChatGPT may be a useful tool for basic healthcare questions, but it is also a fairly rigid response tool that struggles with more complex queries, a new study warns.
More seniors are accepting the role of artificial intelligence in healthcare, particularly as an informational tool. But that makes the possibility of incorrect responses only more worrisome.
Disturbingly, adding factual evidence to a prompt reduced ChatGPT’s accuracy by 17%, the study found.
Not only did ChatGPT fail to give correct or adequate responses to certain, more complicated questions, but the model also gave different responses to the same question when phrased slightly differently, the report said.
Adding an “unsure” option in addition to “yes” or “no” also significantly reduced the accuracy of ChatGPT.
The study authors were careful not to characterize the findings as proof that ChatGPT or AI isn’t useful, but instead that more work needs to be done to refine their utility in settings where the stakes are too high for incorrect or partial answers to be the norm.
“Large language models seem set to be increasingly used for information seeking tasks,” the study authors wrote. “If such tasks drive important health-related decisions (such as what treatment to take) then it is paramount we better understand these models. An understanding of such aspects both helps mitigate potential harms and helps inform the design of better models.”
The concerning findings are part of a recent pattern of research that has addressed AI’s current limitations.
For instance, AI models have been demonstrated to make dangerous mistakes when summarizing hospital discharge notes, McKnight’s recently reported.
Healthcare leaders have warned that current AI tools don’t “know” when they are being trained on biased data, and don’t know when to catalogue responses that might be considered inaccurate.
Please login or register first to view this content.
Copyright © 2024 Haymarket Media, Inc. All Rights Reserved This material may not be published, broadcast, rewritten or redistributed in any form without prior authorization. Your use of this website constitutes acceptance of Haymarket Media’s Privacy Policy and Terms & Conditions.
