(31) Comparison of by-domain performance outcomes within course exams to NAPLEX domains

Objective :

Objective: The purpose of this work is to compare performance of assessment items within course exams with NAPLEX performance domains.

Methods:

Methods: Items in all course exams are tagged to ACPE subdomains and related ability statements using ExamSoft. A crosswalk was created between these ability statements and the six domains of the NAPLEX exam. For students graduating 2023, “by-domain” performance was evaluated for each competency area tested in pre-APPE SOP course exams and compared to each corresponding NAPLEX competency from the 2023 NABP school report. Tagging of assessment items to competency statements commenced with the 2020-2021 academic cycle, limiting the capability to perform “by-domain” analysis for the class of 2023 to assessment within the P2 and P3 curriculum.

Results:

Results: The total number of assessment items tagged to an NABP test domain was 4754. The proportion of assessment items from each performance area within course exams was similar to the % weight for each NAPLEX domain, 1-6, respectively: 14.09% (18%), 13.23% (14%), 36.01% (35%), 11.78% (14%), 12.64% (11%), and 12.24% (7%). Performance was most similar between TTUHSC and NAPLEX exams for Domains 1 and 5 (a difference of 2.37 points and 1.31 points, respectfully) while performance differed the most for Domain 2 (15.32-point difference). Domain 2 was associated with highest performance on course exams but lowest for NAPLEX.

Conclusions:

Conclusions: Though limited to assessing the P2 and P3 curriculum, it was possible to identify areas of the NAPLEX exam that appeared to correlate with performance on related competencies in course exams. The results suggest areas of curriculum and assessment that may require attention for refinement.