156 - From Simulation to Conversation: Using AI to Teach Difficult Conversations

Publication Number: 4153.156

Gillian Brennan, Comer Children's Hospital at University of Chicago Medical Center, Chicago, IL, United States; Lauren Rissman, Advocate Children's Hospital - Park Ridge, Park Ridge, IL, United States; Benjamin T. Conway, Clinical Sim AI LLC, Chicago, IL, United States; Vinod Havalad, Advocate Children's Hospital, Chicago, IL, United States

Background: Delivering difficult news requires nuanced communication skills, yet pediatric trainees often have limited opportunities for structured practice. Traditional simulations are resource-intensive and difficult to scale. Artificial intelligence (AI) offers a potential solution through realistic, on-demand dialogue practice. We developed a prototype conversational chatbot simulating emotionally charged parent interactions and evaluated its acceptability, feasibility, and educational impact among pediatric learners.

Objective: To evaluate the feasibility and acceptability of an AI-powered chatbot designed to help pediatric trainees practice difficult conversations, focusing on changes in confidence, perceived helpfulness, and comfort using AI-based training tools.

Design/Methods: A mixed-methods pre/post pilot survey was administered to seven pediatric intensive care fellows who completed a 15-minute chatbot-based conversation exercise. Surveys assessed (1) confidence having a difficult conversation, (2) perceived helpfulness of chatbot practice, and (3) familiarity and post-session comfort with conversational AI, using 5-point Likert scales (1 = Not at all → 5 = Extremely). Post-survey items also evaluated feasibility, realism, and perceived effectiveness. Data were summarized descriptively; pre/post comparisons used Fisher’s exact tests for improvement in high-category responses (≥ Very/Comfortable).

Results: Confidence improved following chatbot practice, with 71% moderately and 29% slightly confident at baseline shifting to 71% moderately and 29% very confident post-intervention (mean 2.71 → 3.29; p = 0.23). Perceived helpfulness remained high (86% → 100%), while comfort with AI showed the largest gain, rising from 86% not at all/slightly familiar to 71% somewhat/very comfortable (mean 1.86 → 4.00; p = 0.01). Most participants (86%) found the chatbot feasible to integrate into training, 57% rated it realistic and emotionally appropriate, and 86% felt it improved their effectiveness. As shown in Figure 1, Likert distributions shifted rightward across domains, most notably for AI comfort.

Conclusion(s): A brief AI-driven conversation simulation was feasible, well-received, and associated with improved confidence and markedly increased comfort using AI tools. Though confidence gains were not statistically significant, the consistent upward trends highlight the potential of scalable, AI-mediated communication training. Next steps include expanding to a larger, multi-site cohort and assessing long-term retention of communication skills and confidence.

Figure 1.
Figure 1. pre_post_likert_stacked.pdfPre- and Post-Intervention Likert Distributions Showing Shifts in Confidence, Helpfulness, and Comfort with AI