585 - Sherlock and the Machine: A Meta-Cognitive Quest

Publication Number: 2570.585

Sonia Sethi, The Children's Hospital at Monmouth Medical Center, Long Branch, NJ, United States; Renuka Verma, The Children's Hospital at Monmouth Medical Center, Long Branch, NJ, United States

Background: Clinical reasoning and problem solving are essential for reducing diagnostic error. Traditional clinical case conferences including morbidity and mortality require extensive preparation and often offer little protected time for educators. Advances in generative artificial intelligence (AI) create new opportunities to reconstruct cases directly from electronic medical record (EMR) notes, helping learners "think through their thoughts" and reflect on reasoning patterns. These tools can aid in remediating clinical reasoning and identify reasoning gaps in real time.

Objective: 1. Engage pediatricians in diagnostic reflection through structured, audiovisual, AI-assisted cognitive exercises.
2. Provide scaffolded diagnostic cues to allow stepwise reasoning and pave the path for remediation.
3. Incorporate medical humanities principles to foster empathy and provide insight into cognitive bias.

Design/Methods: A 60- minute interactive workshop was created consisting of four learning stations accessed via iPads and QR codes. Each station featured a de-identified pediatric case reconstructed from EMR notes and presented through a large-language-model (LLM)–generated scaffold that highlighted diagnostic reasoning pathways, potential biases, and missed hypotheses. Small groups identified cognitive missteps and discussed remediation strategies. Faculty facilitated discussions while AI generated tailored prompts and summaries. Pre- and post-session surveys measured participants’ confidence in diagnostic reflection, understanding of bias types, and problem-solving techniques.

Results: Fifteen pediatric residents completed the pilot session. Median confidence in diagnostic reflection increased from 2.9 to 4.4 on a 5-point Likert scale (p < 0.05). Participants described the experience as “engaging and eye-opening,” citing AI’s ability to reveal hidden reasoning gaps and deepen diagnostic reflection skills.

Conclusion(s): A metacognitive workshop using AI tools is practical, engaging, and enhances diagnostic reflection skills among pediatric residents. The format is scalable and customizable across diverse academic environments. Future iterations will evaluate integration into residency M&M conferences and diagnostic error reduction curricula.