AI Chatbot Provides "Usually Appropriate" Imaging Recommendations – Fagen wasanni
Fagen Wasanni Technologies
All about XXI Century Technologies
Fagen Wasanni Technologies
Researchers led by Alexander Rau, MD, from the University of Freiburg have developed an AI chatbot that provides “usually appropriate” imaging recommendations based on the American College of Radiology (ACR) Appropriateness Criteria. The chatbot, called accGPT, showed consistently correct answers and contributed to time- and cost-based savings.
Standardized care is needed for efficient and accurate diagnostic imaging, as there is still variability in clinical routines. Some radiologists may lack awareness, and the rapid rise of new imaging technologies and methods, including artificial intelligence (AI), adds to the variability.
While AI tools like ChatGPT have seen increased use in patient-facing settings, researchers have also been exploring their potential as clinical assistance tools. However, these tools have limitations in their training data, which may result in incorrect or incomplete information.
To address this issue, Rau and colleagues incorporated specialized knowledge into their accGPT model. They used the LlamaIndex framework to connect large language models with external data. The accGPT model was tested against benchmarked radiologists of varying experience levels, as well as GPT 3.5 and GPT 4, in adhering to the ACR Appropriateness Criteria.
The researchers found that the accGPT model significantly outperformed radiologists and GPT 3.5 in providing “usually appropriate” recommendations. It also performed better than GPT 4 at a “trend” level. There was no significant difference in radiologists’ ability to give correct answers based on their experience levels.
The accGPT model showed high performance in considering “may be appropriate” recommendations as false recommendations. It also demonstrated high accuracy in considering “may be appropriate” answers as correct.
Additionally, all three chatbots led to significant time and cost savings compared to radiologists. The accGPT model had an average decision time of 5 minutes and a cost of 0.19 euros, while radiologists took 50 minutes and cost 29.99 euros.
The researchers suggested that accGPT could be beneficial to radiologists as an information retrieval tool for rare cases and to ordering physicians as a quick reference guide. Future studies should further assess the costs, availability, and potential radiation dose of the model.
Overall, the accGPT chatbot shows promise in providing accurate imaging recommendations based on established guidelines and has the potential to improve efficiency and decision-making in radiology.
