(45) Enhancing Oncology Infographic Evaluation: A Peer Calibration Approach

Objective : The objective of the current study was to evaluate the effect of student engagement in a peer calibration exercise on drug infographic grading. It was hypothesized that engaging in the exercise would lead to better alignment between students' grades for their peers, as observed throughout the semester, to those of the professors.

Methods: An assignment was generated that required each student to assess select infographics from a previous cohort. The infographics had been scored by two distinct professors, and were selected based on their < 2-point difference between faculty grades. Students were obligated to iterate the assignment, utilizing a predetermined rubric, until they graded 75% of the infographics within a 2-point range of the professors’ evaluation. Subsequently, students were assigned two peer submissions to evaluate across the semester. A comparative analysis of calibrated (n=66) and non-calibrated (n=80) cohorts was performed to ascertain whether the activity brought the peer grades into closer alignment to the professors.

Results: Students across both cohorts (calibrated and non-calibrated) statistically graded higher than the professors on the same infographics. Students in the non-calibrated cohorts showed improvement in understanding grading expectations over the semester, reflected in a decreasing mean difference between faculty and student grading across the semester. A similar trend was observed in the calibrated cohort, but with some distinctions. This cohort exhibited faster learning, with a lower initial (by 18%) and lower overall (by 10%) difference in grading means, compared to the non-calibrated group.

Conclusions: In summary, both cohorts demonstrated a learning process in understanding grading expectations, but the calibrated cohort showed a quicker adaptation and a more pronounced reduction in the difference of means, indicating a potential benefit from calibration.