TOP 34 - Predicting Need for Appendix Ultrasound at Pediatric Emergency Department Triage Using Natural Language Processing

Publication Number: 4737.TOP 34

Haley Thompson, Children's Hospital Los Angeles, Venice, CA, United States; Pradip Chaudhari, Children’s Hospital Los Angeles, Los Angeles, CA, United States; Marianna Costa, University of Padua, Padua, Italy, Santa Barbara, CA, United States; Payal Shah, Children's Hospital Los Angeles, Los Angeles, CA, United States; Amir A. Kimia, Connecticut Children's Medical Center, Boston, MA, United States; Assaf Landschaft, Boston Children's Hospital, Bergisch Gladbach, Nordrhein-Westfalen, Germany; Anna McGinty. Cushing, Children's Hospital Los Angeles, Los Angeles, CA, United States

Background: Appendicitis is the most common pediatric surgical emergency and ultrasound is the first-line test for diagnosis in the emergency department (ED). Early diagnostic ordering at the time of triage may reduce ED length of stay (LOS) and shorten time to diagnosis, but current strategies such as nursing order sets may have inconsistent implementation and can lead to over testing. Automated diagnostic ordering algorithms are a promising alternative to improve care efficiency without unnecessary testing. A prior machine-learning model incorporating natural language processing (NLP)-derived features showed strong performance in predicting appendix ultrasound ordering but relied on variables identified from data patterns that may not reflect typical clinical decision-making. Deriving models that restrict inputs to evidence-based, a priori clinical predictors may improve interpretability and clinician trust in such tools.

Objective: To derive an NLP-enriched predictive model using a-priori clinical variables available at the time of ED triage and evaluate model performance to predict (1) appendix ultrasound order and (2) appendix ultrasound order with subsequent positive appendicitis diagnosis. Additionally, to (3) describe theoretical acceleration in care associated with applying the model at time of ED triage.

Design/Methods: We will conduct a retrospective cross-sectional study of all ED encounters for children ages 0-17 years from October 2015-June 2024 at an academic children's hospital. We will derive a logistic regression model and evaluate its performance for each outcome of interest: (1) appendix ultrasound ordered and (2) ultrasound ordered with subsequent positive appendicitis diagnosis. Predictors will include structured demographic and clinical variables available at time of triage and NLP-derived features extracted from triage nurse notes. NLP predictors will be defined a priori based on clinically relevant features identified in prior literature. NLP analysis will be performed using Document Review Tool (DrT) software. Model performance will be evaluated using area under the receiver operating curve (AUROC). Model tuning will prioritize higher specificity and positive predictive value to minimize unnecessary diagnostic utilization. Potential clinical impact will be estimated by calculating the theoretical acceleration in care if ultrasound was ordered at triage (Figure 1). This study was approved by the lead author's institutional review board. NLP analysis is expected to be complete by December 2025 with predictive model completion in January 2026.

Figure 1: Comparison of current and proposed workflows for early diagnostic ordering using predictive models