(257) Using a Modified Delphi Panel to Develop Best Practices In Course Evaluations of Teaching

Objective : Course evaluations are widely used in academia to gather feedback about teaching and learner experiences. A recent national survey revealed many differences in developing, administering, and utilizing course evaluations among pharmacy programs. Thus, the objective of this project was to generate consensus on best practices in creating, administering, and using student course evaluations.

Methods: A modified delphi approach with up to three rounds of a survey with 10-15 panelists was planned; consensus was set at 70% agreement. A list of experts (assessment, course evaluation) from different: roles (assessment leads, non-administrative faculty, administrators), geographic regions, and institutional types (public, private) were identified and invited to be panelists. Survey items related to the study objective were generated from prior work and the literature, reviewed by the research team after each round, and adjusted until consensus was reached by the panelists. Three reminders were sent to increase the response rate.

Results: Consensus was achieved in the second round with 14 panelists. Best practices for creating and administering course evaluations of teaching (100% agreement) were: create a standard set of questions, base rating scales on principles of evaluation and measurement, have a rationale before making them mandatory, and timely administration to students who have recently experienced the course. Best practices that met consensus, but had a lower level of agreement were: evaluations should assess the quality of instruction, the fairness of course assessments, and benchmarks should be used to interpret results.

Conclusions: As programs seek to balance student input with other measures of course and teaching quality, identifying and using best practices may aid programs in enhancing the utility of course evaluations.