298 - Impact of a Machine Learning–Based Decision Support Tool on Pediatric Emergency Medicine Physician Diagnosis of Kawasaki Disease

Publication Number: 1284.298

Michael A. Gardiner, University of California, San Diego School of Medicine, San Diego, CA, United States; Jonathan Lam, University of California, San Diego School of Medicine, Fremont, CA, United States; shamim Nemati, University of California, San Diego School of Medicine, San Diego, CA, United States; Jane Burns, UCSD, La Jolla, CA, United States; Adriana Tremoulet, University of California San Diego, La Jolla, CA, United States

Background: Kawasaki Disease (KD) is the leading cause of acquired heart disease in children in developed countries. Diagnosis relies on clinician interpretation of clinical and laboratory findings, as there is no definitive diagnostic test. To address this diagnostic challenge, our group developed and validated Kawasaki MATCH, a machine learning-based clinical decision support tool.

Objective: To assess the effect of Kawasaki MATCH on pediatric emergency medicine (PEM) physician accuracy in diagnosing and managing KD.

Design/Methods: PEM physicians from a single academic center reviewed 49 case vignettes (24 KD, 25 febrile controls) derived from emergency department notes of patients in an existing KD database. All KD cases met American Heart Association criteria for complete or incomplete KD, while controls had ≥3 days of fever and ≥1 KD clinical feature. Each vignette included clinical history, vital signs, physical examination findings, and laboratory results. The Kawasaki MATCH tool generated a KD probability score using the same data, applying a validated diagnostic threshold of ≥40%. Physicians reviewed vignettes in randomized order and classified each as KD or not-KD, rated the likelihood of KD, and determined patient disposition (admission vs. discharge). After a three-month washout period, physicians reassessed the same vignettes with Kawasaki MATCH decision-support results provided. Diagnostic accuracy, net reclassification index (NRI), and McNemar's test were used to evaluate performance.

Results: Twenty-one PEM physicians completed both assessments. Overall diagnostic accuracy improved from 67.3% to 73.4% with Kawasaki MATCH support. Sensitivity increased, with a small, nonsignificant decrease in specificity. NRI was 0.17 for KD cases and −0.05 for controls. The odds of accurate diagnosis increased 1.7-fold overall (95% CI, 1.30-2.24) and nearly four-fold for KD cases (OR 3.87; 95% CI, 2.57-5.99). Planned admissions for KD patients rose from 83% to 98%, while admissions for controls were unchanged (39%). The greatest improvement occurred among providers with the lowest baseline accuracy.

Conclusion(s): Use of Kawasaki MATCH enhanced PEM physician accuracy and sensitivity for KD diagnosis, significantly reducing inappropriate emergency department discharges of KD patients. The tool had minimal effect on control patient disposition and demonstrated the largest benefit among lower-performing providers. Kawasaki MATCH shows strong potential as a clinical decision support and screening tool for KD in the emergency department setting.

Table 1: Characteristics of Pediatric Emergency Medicine Physicians Completing Case Vignettes


Table 2: Performance Characteristics of unaided PEM physicians, physicians with Kawasaki MATCH decision support, and the Kawasaki MATCH algorithm alone in predicting KD. Median (IQR)


Table 3: Odds Ratio for Accurate Diagnosis with the addition of Kawasaki MATCH decision support