(277) Integrating ChatGPT in a Second-Year Drug Information Course at the Gregory School of Pharmacy

Objective : ChatGPT is an artificial intelligence (AI) tool that has prompted discussion for its use in pharmacy education and healthcare. ChatGPT-3.5 was intentionally integrated into a second-year DI course for the first time in Fall 2023 at the Gregory School of Pharmacy. The objective was for students to evaluate the accuracy, completeness, and elegance of information provided by ChatGPT-3.5 when compared with standard DI resources.

Methods: The DI course employs a process-oriented guided inquiry learning approach. Students in six groups of five, evaluated responses generated from ChatGPT for: depth of information; relevance to DI inquiry and/or omission of content; and its ability to articulate an intelligent response. Ten questions throughout four different modules were assigned during in-class exercises. Students submitted their responses via an online survey using a 5-point Likert scale (1-poor to 5-excellent). Students also provided feedback on the process via course evaluations. Descriptive statistics were used. The project was deemed institutional review board exempt.

Results: Among the ten questions, the average ratings for accuracy, completeness, and elegance using ChatGPT were 3.4; 3.2; and 4.5, respectively. Questions related to cost, drug of choice, and ongoing studies were rated the lowest for accuracy and completeness. The patient education question received the highest ratings overall. In course evaluations, students reflected on the role of pharmacists, as the medication expert, to verify information from AI.

Conclusions: Access, capability, and familiarity with AI tools will increase and exposure is valuable in training student pharmacists. When compared with standard drug references, students detected limitations in the accuracy and completeness of ChatGPT-3.5’s responses across several categories. This exercise reinforces that pharmacists must be adapt at technology and authenticate medical information generated from AI tools.