133 - Problem- vs. Systems-Based Clinical Documentation on Measures of Clinical Reasoning, Diagnostic Accuracy, and Mental Demand Among Residents

Publication Number: 4130.133

Laura O'Neill, Children's National Hospital, Alexandria, VA, United States; Kristen Johnson, George Washington University School of Medicine and Health Sciences, Fairfax, VA, United States; Adam Cohen, Baylor College of Medicine, Houston, TX, United States; Michael Perry, Nationwide Children's Hospital, Pataskala, OH, United States; Rebecca Trillo, Baylor College of Medicine, Houston, TX, United States; Andrew P.J.. Olson, University of Minnesota Medical School, Minneapolis, MN, United States

Background: Clinical documentation is a reflection of clinical reasoning. The use of system- vs problem-based plan structure varies by preference and contextual factors. Limited evidence suggests that problem-based documentation may better support clinical reasoning and diagnostic accuracy.

Objective: Explore the association of note structure and clinical reasoning performance in resident trainees.

Design/Methods: Pediatric and Medicine-Pediatric residents from four U.S. residency programs read four published case stems from Pediatrics in Review and write an assessment and plan as well as report a working diagnosis, confidence in diagnosis (0-100), and mental demand (0-100) for each case. Plan structures are then labeled as problem-based, system-based, mixed structure, or unstructured. Case responses without plans are excluded. Primary outcome is the averaged Revised-IDEA (Interpretative summary, Differential diagnosis, Explanation of reasoning for lead and Alternative diagnoses) score, a validated measure of clinical reasoning, as rated by 2 co-authors (score range 0-10). Secondary outcomes are accuracy in diagnosis (as rated by 2 co-authors), confidence in diagnosis, and mental demand. Kruskal-Wallis tested for associations between plan type and R-IDEA score or mental demand. ANOVA was used for outcome of confidence and a proportional odds model for diagnostic accuracy.

Results: 32 participants (Table 1) completed 98 total case responses, though 46 responses are included in this analysis as 9 were excluded due to no plan and the remainder of responses still require R-IDEA scoring. 30% (n=14) of case responses used problem-based structure, 10% (n=5) used systems-based, and 60% (n=28) were unstructured. R-IDEA scores differed by plan type (Kruskal-Wallis test chi-squared = 6.356, p = 0.04) with system-based plans receiving higher scores, although pairwise Dunn's test showed no significant differences when adjusting for multiple comparisons (Figure 1). There was no association between plan type and the secondary outcomes (Figure 2).

Conclusion(s): This smaller study did not find statistically significant differences in clinical reasoning scores based on plan type or the secondary outcomes. However, there was a trend for system-based plans to have higher clinical reasoning scores. There are several limitations of this study, including the small sample size leading to potential underpowering and lack of participant effort in simulated cases. Next steps include scoring additional case responses for inclusion in the final analysis and applying these methods to authentic clinical documentation.

Table 1. Demographic information for the participants (n=25) who completed full case survey
Partial responses to the case survey did not include demographic information.

Figure 1. Box plot demonstrating R-IDEA scores based on plan type
Box plot showing the interquartile ranges and median score. The x-axis is plan type and y-axis is Revised-IDEA score (range 0-10). Plans with no structure utilized lists, bullet points, dashes, etc. The legend shows resident training year by color. Each color dot on the box plot is a unique participant response.

Figure 2: Box plots demonstrating no differences in use of plan types by participant ratings of confidence in diagnosis or mental demand of case
Box plot showing the interquartile ranges and median scores. The x-axis is plan type and y-axis is confidence in diagnosis and mental demand by case. Plans with no structure utilized lists, bullet points, dashes, etc. The legend shows resident training year by color. Each color dot on the box plot is a unique participant response.