(148) Validation of a Patient Education Skills Assessment Rubric for Initial Continuous Glucose Monitor Placement

Objective : To validate an assessment rubric for an initial continuous glucose monitor (CGM) placement patient education session by measuring the interrater reliability (IRR) to ensure consistency when evaluating student performance.

Methods: Second or third-year pharmacy students at two institutions were evaluated on their ability to counsel a patient on the initial use of a CGM device (Dexcom G6 or Libre 2) during a practical assessment. Faculty evaluators used a standardized rubric to evaluate students during the assessment. Upon quarter completion, two additional evaluators, who were not involved in the courses, reviewed a recording of each session and evaluated the students using the same rubric. These additional evaluations had no effect on student grades. The intraclass correlation coefficient (ICC2k) was calculated to determine the IRR of the rubric overall and its subcomponents of competency and etiquette.

Results: A total of 53 students completed this counseling session with a video recording. Each student was graded using the CGM rubric by three total faculty evaluators. The average student score was 23.6 out of 25 possible points. Average scores were slightly lower in students counseling on the Dexcom G6 device compared to Libre 2 (22.51 vs. 23.85, p=.005). The ICC2k value among three evaluators was high [0.87, 95% CI (0.80-0.91)], indicating a high level of agreement for the total rubric score.

Conclusions: The grading rubric to assess students’ ability to counsel patients on CGM device initiation demonstrated high reliability among multiple evaluators and may be a useful tool for institutions to implement as part of CGM counseling activities.