379 - Leveraging machine learning for prediction of grade 3 bronchopulmonary dysplasia (BPD) or death in preterm infants using Epic Cosmos data

Publication Number: 1363.379

Timothy Nelin, Childrens Hospital of Philadelphia, Philadelphia, PA, United States; Sara Handley, Childrens Hospital of Philadelphia, Philadelphia, PA, United States; Kristan Scott, Childrens Hospital of Philadelphia, Philadelphia, PA, United States; Jennifer F. Culhane, Yale School of Medicine, New Haven, CT, United States; Jay Greenspan, Sidney Kimmel Medical College at Thomas Jefferson University, Philadelphia, PA, United States; Heather Burris, Children's Hospital of Philadelphia, Philadelphia, PA, United States; Kevin Dysart, Children's Hospital of Philadelphia, Philadelphia, PA, United States

Background: BPD is the most common chronic morbidity of preterm birth. Infants with grade 3 BPD, defined by invasive mechanical ventilation (IMV) at 36 weeks' postmenstrual age (PMA), experience the highest risk for long-term adverse outcomes. Early prediction of grade 3 BPD or death may improve risk stratification and guide targeted intervention.

Objective: To assess characteristics associated with grade 3 BPD/death and use machine learning to predict infants at risk using data through the first 21 days after birth.

Design/Methods: Using Epic Cosmos, we identified infants born at < 32 weeks' gestation (2018-2024) with birthweight >200g, weight at discharge/death < 10kg, and survival ≥21 days. Grade 3 BPD was defined as a BPD diagnosis with IMV or ventilator charges beyond 36 weeks PMA. We trained logistic regression, random forest (RF), and XGBoost models using sequential feature sets: (1) maternal, (2) maternal + infant birth, and clinical data from days (3) 0-1, (4) 2-7, (5) 8-14, and (6) 15-21. Each successive model built upon the prior feature set. Models were run untuned and tuned using (a) 85% sensitivity and (b) F1-threshold strategies.

Results: Among 88,912 infants, 8,629 (11.8%) developed grade 3 BPD/death (Table 1). Infants with grade 3 BPD/death were more likely to be male, lower birthweight and gestational age, and had higher rates of IMV, acidosis, and hypercapnia in the first 21 days. Model discrimination, assessed by area under the receiver operating curve (AUC), improved substantially after adding infant birth characteristics (AUC 0.50 maternal vs 0.82 maternal + infant) and plateaued thereafter (AUC 0.83-0.86 across 0-21 days) (Figure 1). At 15-21 days, the untuned RF model achieved AUC 0.86, accuracy 0.90, sensitivity 0.02, specificity 0.99, positive predictive value (PPV) 0.60, and negative predictive value (NPV) 0.91, indicating strong overall fit but poor case detection. The F1-tuned RF model provided improved balance (AUC 0.86, accuracy 0.84, sensitivity 0.67, specificity 0.85, PPV 0.33, NPV 0.96). Persistent low pH and elevated pCO2 through 21 days and IMV at days 0-1 further contributed to model importance (Figure 2).

Conclusion(s): Machine learning models using real-world electronic health record data modestly predicted grade 3 BPD or death by 36 weeks' PMA. Model performance improved significantly after the inclusion of birth data, highlighting gestational age and birth weight as dominant predictors. Modest performance gains following inclusion of clinical indicators underscore the multifactorial nature of BPD and need for precision phenotyping and novel biomarkers to enhance prediction.

Table 1. Selected cohort characteristics.
Abbreviations: BPD, bronchopulmonary dysplasia; SVI, social vulnerability index; pH, hydrogen ion concentration; pCO2, partial pressure of carbon dioxide; WBC, white blood cell.
Definitions: Grade 3 BPD was defined as a BPD diagnosis with IMV ≥ 36 weeks PMA or ventilator charges extending beyond 36 weeks. Low pH was defined as pH < 7.0 on days 0-1 and < 7.2 on days 2-21. High pCO2 was defined as > 70 mm Hg on any day within the interval. Abnormal WBC was defined as < 5 × 10^9/L or > 35 × 10^9/L.

Figure 1. Receiver operating characteristic (ROC) curves for sequential machine learning models predicting grade 3 bronchopulmonary dysplasia (BPD) or death among infants <32 weeks' gestation using Epic Cosmos (2018-2024).
Panels A-F show untuned models incorporating progressively more data: (A) maternal characteristics; (B) maternal + infant birth characteristics; (C-F) addition of clinical data from days 0-1, 2-7, 8-14, and 15-21. Panels G and H display tuned models at 15-21 days using 85% sensitivity and F1-threshold strategies. Logistic regression (GLM), random forest (RF), and XGBoost (XGB) models were compared at each step. Discrimination improved after adding birth characteristics (AUC 0.50 to 0.82) and plateaued thereafter (AUC 0.83-0.86 across 0-21 days). At 15-21 days, positive predictive values (PPV) remained modest (0.23-0.60) given the outcome prevalence of 11.8%, while negative predictive values (NPV) were high across all models (0.91-0.98). Untuned models demonstrated high overall accuracy (0.90-0.91) but poor sensitivity (GLM 0.17, RF 0.02, XGB 0.14). The untuned RF model achieved AUC 0.86, accuracy 0.90, sensitivity 0.02, specificity 0.99, positive predictive value (PPV) 0.60, and negative predictive value (NPV) 0.91. The F1-tuned model provided balanced performance (AUC 0.86, accuracy 0.84, sensitivity 0.67, specificity 0.85, PPV 0.33, NPV 0.96), while the 85%-sensitivity-tuned model increased case detection (AUC 0.86, accuracy 0.74, sensitivity 0.85, specificity 0.72, PPV 0.25, NPV 0.98). The plateau in AUC after birth data underscores the dominant influence of perinatal characteristics despite the addition of physiologic variables.

Figure 2. Random forest permutation importance for prediction of grade 3 bronchopulmonary dysplasia (BPD) or death by 36 weeks' postmenstrual age.
Permutation-based out-of-bag (OOB) importance estimates identify the most influential predictors in the final 15-21-day random forest model. Maternal, perinatal, and early postnatal variables were: maternal age, race and ethnicity, insurance type, rural-urban classification, social vulnerability index, smoking, antenatal steroid exposure, gestational hypertension, gestational diabetes, pre-existing hypertension, obesity, and diabetes; gestational age, birth weight category, sex, and small-for-gestational-age status; and serial physiologic and laboratory indicators of clinical status across days 0-1, 2-7, 8-14, and 15-21 of life - specifically low pH, high pCO2, abnormal white blood cell count ( <5 or >35 × 109/L), and exposure to invasive ventilation. The ten variables with highest permutation importance were low pH (15-21 days), high pCO2 (15-21 days), low pH (8-14 days), low pH (2-7 days), birth weight 501-750 g, high pCO2 (8-14 days), high pCO2 (2-7 days), invasive ventilation (0-1 days), gestational age 24 weeks, and gestational age 23 weeks. Collectively, these findings underscore the roles of persistent respiratory acidosis and extreme prematurity in predicting severe BPD or death despite broad inclusion of maternal and perinatal factors.