511 - Identifying Children with Medical Complexity Using an Electronic Medical Record-Sourced Data Repository

Publication Number: 1491.511

Lara Bruno-Boucher, University of Toronto Temerty Faculty of Medicine, Toronto, ON, Canada; Sara Santos, Hospital for Sick Children, Toronto, ON, Canada; Hallie Benjamin, The Hospital for Sick Children, Toronto, ON, Canada; Bianca Arenas Rodriguez, The Hospital for Sick Children, Scarborough, ON, Canada; Joanna Soscia, The Hospital for Sick Children, Toronto, ON, Canada; Kimberly Colapinto, The Hospital for Sick Children, Toronto, ON, Canada; Susan Miranda, The Hospital for Sick Children, Toronto, ON, Canada; Julia Orkin, The Hospital for Sick Children, Toronto, ON, Canada; Peter J. Gill, The Hospital For Sick Children, Toronto, ON, Canada; Alene Toulany, The Hospital for Sick Children, Toronto, ON, Canada; Lillian Sung, The Hospital For Sick Children, Toronto, ON, Canada; Eyal Cohen, The Hospital for Sick Children, Toronto, ON, Canada

Background: The use of electronic medical record (EMR) algorithms to identify non-diagnostically defined populations has potential broad utility. One application is to identify children with medical complexity (CMC). Clinical initiatives to identify CMC through claims-sourced databases lack sufficient granularity for some criteria (e.g. functional limitations), while those based on referrals may create selection bias. CMC approaching the age of transfer to adult care face challenges such as difficulty accessing adult care teams. As part of a program to support transition to adult care for CMC, we aimed to develop an EMR algorithm for ascertainment.

Objective: (1) Develop an EMR algorithm to identify CMC approaching the age of transfer to adult care (age 17 to < 18 years); and (2) validate this algorithm in children aged 1 to < 17 years.

Design/Methods: Approved by the SickKids Research Ethics Board (#3770). This is a cross-sectional study of an algorithm developed iteratively to identify CMC from the SickKids Enterprise-wide Data in Azure Repository, a validated schema using Epic data. Adopting the Complex Care for Kids Ontario standard criteria for CMC; (1) complexity (multiple care teams, family circumstances), (2) chronicity (≥ 6 months or life expectancy < 6 months), (3) fragility (severe condition, risk of negative health outcomes), and (4) technology dependance/high-intensity care (medical devices, great level of care), we developed an algorithm to identify patients with a 1-year lookback window, refining criteria after each iteration to maximize positive predictive value (PPV; see Table 1). In Phase 1, we compared youth aged 17 to < 18 years identified by each iteration to chart review by two independent reviewers. In Phase 2, we validated the final algorithm in a separate cohort (1 to < 17 years).

Results: 280 charts were reviewed in Phase 1. Algorithm Version 6 performed best, identifying CMC with a PPV of 85.5% (95% CI 73.3%-93.5%). The validation search (947 charts: 671 already enrolled in a referral-based complex care program, 276 requiring manual review) performed similarly, identifying CMC with a PPV of 89.1% (95% CI 86.9%-91.0%). In a 10% sample of charts requiring manual review (n=28), interrater agreement was 89.3% (95% CI 71.7%-97.7%); Cohen's κ = 0.73 (95% CI 0.43-1.0).

Conclusion(s): We developed a valid EMR algorithm to identify CMC transitioning to adult care for recruitment to a targeted intervention. The algorithm performed similarly in a broader set of patients, holding promise for use across multiple purposes, including program recruitment, quality improvement, and population health monitoring.

Table 1. Comparison of accuracy in identifying children with medical complexity for each iteration of the algorithm.
Searches were run with an index date of August 31, 2024.

Figure 1. Complex Care for Kids Ontario framework for children with medical complexity (CMC), including its four domains within the inner ring and the associated electronic medical record data elements used for ascertainment in the outer ring.
A fifth category (geography; not illustrated) was included for patients that met criteria in 3 of the 4 categories, using postal code data to define rurality. Patients who then met geography criteria were also classified as CMC.

Figure 2. Screening process for the validation cohort (aged 1 to <17 years).
CCKO = Complex Care for Kids Ontario.