200 - Asthma and Chest X-Rays in Children: Why NLP Outperforms ICD-10 Codes in Measuring Utilization and Value

Publication Number: 3192.200

Alek C. Jorge, Connecticut Children's Medical Center, Berlin, CT, United States; Alexander H. Hogan, Connecticut Children's, Hartford, CT, United States; Assaf Landschaft, Boston Children's Hospital, Bergisch Gladbach, Nordrhein-Westfalen, Germany; Annina Beus, Connecticut Children's Medical Center, Hartford, CT, United States; Noah Sobel, University of Connecticut School of Medicine, Farmington, CT, United States; Ilana Waynik, Connecticut Children's Medical Center, West Hartford, CT, United States; Richelle deMayo, University of Connecticut School of Medicine, Westport, CT, United States; Amir Kimia, Connecticut Children's Medical Center, Boston, MA, United States

Background: Asthma-related ICD-10 codes are structured data elements often used as proxies for clinical asthma presentations to (A) assess chest X-ray (CXR) ordering rates in efforts to reduce unnecessary imaging in children, and (B) evaluate CXR diagnostic yield in emergency department (ED) visits. Because codes are assigned after visits, we hypothesized they may not accurately reflect clinical narratives in the electronic health record (EHR) and may overrepresent visits with negative CXRs. Clinical narratives offer added value by identifying patients with prior CXRs and capturing factors influencing imaging decisions, such as previous antibiotic use or confirmed viral infections (e.g., influenza).

Objective: To compare the performance of ICD-10 codes with clinical narratives extracted from the EHR.

Design/Methods: We applied natural language processing (NLP), combining a support vector machine (SVM) classifier with a fine-tuned Clinical-Longformer model, to identify children aged 2-18 years presenting with respiratory symptoms at a tertiary care center during the second half of 2024. Inclusion criteria are summarized in Table 1. Sensitivity was set at ≥95%. Two asthma-related ICD-10 code sets were analyzed: (A) primary diagnoses per Low Value Care (LVC) calculators, and (B) any asthma-, wheeze-, or reactive airway-related codes assigned during the visit. Reviewers blinded to ICD-10 coding manually validated NLP outputs. Categorical variables were compared using chi-square tests.

Results: NLP identified 1,565 children (sensitivity 97.6% [95% CI: 95.3-99.9%], specificity 66.9% [95% CI: 58-75%]); 20 additional cases (1.2%) were found via ICD-10 codes only, totaling 1,585 patients. A primary asthma diagnosis (per LVC) was assigned in 399 cases; 715 children had any asthma-related diagnosis. ICD-10 code performance metrics appear in Table 2. CXR ordering rates and diagnostic yield were lower for asthma visits identified by ICD-10 codes versus clinically confirmed asthma-related visits (Table 3).

Conclusion(s): ICD-10 codes alone inadequately assess CXR use in pediatric asthma. NLP analysis of EHR narratives better captures practice patterns, identifies truly low-value scenarios, and supports targeted interventions, avoiding risks of missed pneumonia, provider resistance, and patient or family dissatisfaction.

Table 1. Clinical presentation of asthma where CXR may be considered, and exclusion criteria.
* Positive Pertussis, RSV, Influenza, or other respiratory pathogens

Table 2. Performance of ICD-10 codes compared with the clinical narrative.


Table 3. Test ordering and yield among children identified with clinical presentation of asthma, based on chart review (clinical narrative).
* We considered a positive CXR positive for pneumonia, if the provider chose to treat it with antibiotics