Elizabeth Briand
Every day seems to bring with it another new use for ChatGPT, the artificial intelligence (AI)–driven chatbot created by OpenAI. Experts have speculated that AI could help pharmacists improve safety, predict health risks, and more. A new study published in the November/December 2023 issue of JAPhA suggested that ChatGPT might one day play a significant role in medication therapy management (MTM).
Researchers of the study sought to determine whether ChatGPT could be used to identify medication interventions by integrating patient and drug databases—and specifically to see how effective it was in MTM for simple, complex, and very complex cases.
Putting AI to the test
Investigators took 39 sample cases—with 13 each designated as simple, complex, and very complex by a team of clinical pharmacists—and assessed ChatGPT’s responses to each case based on three criteria: its ability to identify the interactions between drugs, diseases, substances, and supplements; its precision in recommending alternatives; and its appropriateness in devising management plans. Using a predetermined formula, ChatGPT’s responses were considered valid if it could accurately provide 70% of the clinical points from each criterion when compared to the previously researched answers. This threshold was based on the passing score requirements for the United States Medical Licensing Examination and NAPLEX.
ChatGPT’s responses earned a 39 out of 39. It did very well in the simple and complex cases, but the very complex cases required more informational support for it to achieve the correct answers. Overall, the study found that “ChatGPT was able to identify potential interactions and provide reasonable management plans, but limitations existed in recommending alternative medical therapy and specifying medication recommendations.”
Don Roosan, PharmD, PhD, lead author of the study from Western University of Health Sciences in California, who is a proponent of the use of informatics and technology in medicine, was surprised at the full spectrum of results. “One of the reasons MTMs are so unique is because they take years of experience [to do them well],” he said. For ChatGPT, which is continuing to evolve every day, to achieve a perfect score was eye-opening.
“Achieving a 100% success rate in simple cases was expected to some extent, given the access to all available online databases, but maintaining a commendable accuracy even in complex and very complex cases was particularly surprising,” said Yanting Wu, PharmD, a postdoctoral fellow with Indiana University School of Medicine’s Division of Clinical Pharmacology, who also worked on the study.
Looking at potential benefits down the road
Findings of the study suggest that in the not-too-distant future, the use of ChatGPT in MTM could help enhance patient safety, lower health care costs, and help providers identify potentially harmful drug interactions. These applications, in turn, could assist increasingly busy health care providers and pharmacists by decreasing the time needed to create MTM plans.
“Beyond this, patients stand to gain additional benefits, experiencing more personalized and effective MTM,” Wu said. “By analyzing extensive datasets, ChatGPT can deliver tailored recommendations based on individual patient characteristics, leading to improved treatment outcomes, reduced risks of adverse effects, and enhanced overall health.”
AI may also benefit patients directly. “Medical knowledge and data have been kind of a monopoly where only doctors and pharmacists can access that information,” Roosan said. “This can help empower patients to understand [their care]. They don’t have to Google something and get gibberish. They can ask questions and get information.”
The study also acknowledged a number of hurdles that must be overcome before using ChatGPT for MTM. “It’s crucial to acknowledge the challenges and limitations for using AI tools in health care, particularly regarding ethical considerations, privacy concerns, and the need for continuous refinement and integration into existing health care systems,” said Wu.
ChatGPT engages in MTM by collecting and processing sensitive patient data. Because of that, the study authors noted, “opportunities for data breaches and unauthorized access to this information could compromise patient privacy and confidentiality.”
In addition, ChatGPT’s accuracy and consistency could be elevated even further by giving it access to more major reference management materials such as Lexicomp, building on the data already readily available to it online.
While there is still much work to be done, Roosan believes in the value of AI as a potentially valuable tool in health care. “Large language-based models are completely going to transform medicine, nursing, and pharmacy,” Roosan said. “We can’t fear this technology, we have to embrace it.” ■