717 - From Trainee to Trusted: Validity and Reliability of EPA Assessments in the Pediatrics Clerkship

Publication Number: 2700.717

Meenakshi Chandrasekaran, University of Minnesota Medical School, Minneapolis, MN, United States; Julia Lister, University of Minnesota Medical School, Minneapolis, MN, United States; Rachel Poeppelman, University of Minnesota, Minneapolis, MN, United States; Claudio Violato, University of Minnesota Medical School, Minneapolis, MN, United States; Kathleen Lane, University of Minnesota Medical School, Minneapolis, MN, United States

Background: Entrustable professional activities (EPAs) have been proposed as a holistic approach to competency-based assessment. The 13 Core EPAs for Entering Residency (CEPAER) are essential tasks that a medical student should be trusted to perform with indirect supervision upon entering residency, based on demonstrated competence.

Objective: To study the validity and reliability of workplace-based assessments of the 13 core EPAs as measurements of medical student performance and growth over the Pediatrics clerkship.

Design/Methods: A total of 641 third-year medical students at the University of Minnesota Medical School participated. Students were enrolled in a required four-week Pediatrics clerkship during 3 academic years (2022-2025) and assessed across the CEPAERs. Regression analyses were employed to examine if the growth curve of each of the EPAs follow a curvilinear structure as hypothesized in negative exponential learning theory. The reliability of the EPA assessments was estimated using two-facet generalizability analysis defined as Ep2 coefficients.

Results: There were 9,196 EPA-based assessments (mean = 14.3 [SD=4.6] assessments per student), provided by 553 assessors; 348 (63%) residents and 205 (38%) faculty members. The growth curves for most EPAs ratings followed the predicted negative exponential curve. The slope of the growth curves showed variation by EPA. Adequate reliability of ratings was achieved with 5 raters assessing on 5 assessment occasions (Ep2 coefficient ≥ 0.70).

Conclusion(s): EPAs represent a valid measure for medical student growth during the Pediatrics clerkship, and adequate reliability was achieved with 5 raters assessing students on 5 occasions.