45 - Validation of an Algorithm to Identify Infants With Invasive Bacterial Infections Using Administrative Data: A Multicenter Retrospective Cohort Study

Publication Number: 3042.45

Pratikshya Karki, Golisano Children's Hospital at The University of Rochester Medical Center, Rochester, NY, United States; Jeffrey P. Yaeger, Golisano Children's Hospital at The University of Rochester Medical Center, Rochester, NY, United States; Melissa Knudson, Wake Forest Baptist Health - Brenner Children's Hospital, Winston Salem, NC, United States; Andrew W. Osten, Upstate Golisano Children's Hospital, Syracuse, NY, United States; Heather Territo, Jacobs School of Medicine and Biomedical Sciences at the University at Buffalo, Buffalo, NY, United States; Christopher Woll, Albany Medical Center, Albany, NY, United States; Mya Dickerson, Golisano Children's Hospital at The University of Rochester Medical Center, Rochester, NY, United States; Mustafa Mahmood, Albany Medical College, Albany, NY, United States; James M. Cheung, State University of New York Upstate Medical University, Syracuse, NY, United States; Morgan Mitchell, Jacobs School of Medicine and Biomedical Sciences at the University at Buffalo, Buffalo, NY, United States

Background: Because invasive bacterial infections (IBIs; bacteremia, bacterial meningitis) occur in only 2-3% of young, febrile infants, large samples are needed to examine IBI epidemiology and outcomes with sufficient statistical power. The current diagnostic gold standard relies on manual review of blood and cerebrospinal fluid (CSF) cultures in identifying IBIs which is time and resource intensive. Administrative data offer a scalable alternative, but the performance is unknown for using diagnosis codes to identify infants with IBIs.

Objective: To validate the performance of an algorithm combining IBI diagnosis codes with length-of-stay (LOS) ≥4 days in identifying culture-confirmed IBIs.

Design/Methods: We conducted a multicenter retrospective cohort study across five children's hospitals from January 2016 through December 2023. We included infants aged 8-364 days with a blood and/or CSF culture obtained and LOS ≥4 days. The gold standard for IBI diagnosis was adjudicated blood and CSF culture results. Using shared definitions and procedures, each site extracted electronic health record data to identify IBIs using the gold standard and algorithm, respectively. To examine algorithm performance, we estimated the area under the receiver operating characteristic curve (AUC), sensitivity, specificity, positive predictive value, and negative predictive value in identifying culture-confirmed IBIs. We further explored algorithm performance by restricting the sample to infants aged 8-60 days (Sub-analysis 1) and modifying algorithm LOS thresholds (i.e., ≥5, ≥6, and ≥7; Sub-analysis 2).

Results: Among 2,498 infants with blood and/or CSF culture (Table 1), the algorithm demonstrated an AUC of 0.90 (95% CI, 0.87-0.92), a sensitivity of 80.6% (95% CI, 75.2%-85.3%), and a specificity of 98.6% (95% CI, 98.0%-99.1%). See Table 2 for complete performance characteristics. Site-specific sensitivities varied (66.2%-96.3%) as specificities remained high (95.2%-99.6%). We observed similar results when restricting the sample to infants aged 8-60 days (Table 3). Sensitivities progressively declined with longer LOS thresholds whereas specificity remained stable.

Conclusion(s): In this multi-site validation study, an algorithm combining IBI diagnosis codes with LOS ≥4 days identified febrile infants with culture-confirmed IBIs with high sensitivity and specificity. Findings support the use of the algorithm to accurately identify infants with IBIs, enhancing the capabilities to study infants at risk for IBIs using administrative data.

Table 1. Characteristics of Subjects Included in the Primary Analysis (LOS ≥ 4 Days; N = 2,498)


Table 2. Performance of the Algorithm in Identifying Culture-Confirmed Invasive Bacterial Infections, Pooled and by Site (% [95% CI])


Table 3. Secondary Analyses: Algorithm Performance by Age and LOS Threshold (% [95% CI])