160 - Concordance Between Pediatric Safety Indicator Measure Sets Across a Sample of Pediatric Hospitals

Publication Number: 3154.160

Phillip D.. Hahn, Boston Children's Hospital, Boston, MA, United States; Al Ozonoff, Harvard Medical School, Boston, MA, United States; Carly E.. Milliren, Boston Children's Hospital, Boston, MA, United States

Background: Medical errors are frequent in inpatient pediatric hospital settings and are associated with increased hospital costs, lengths of stay, and incidence of morbidity and mortality. The Centers for Medicare and Medicaid Services Hospital-Acquired Conditions (HAC) Reduction Program and the Agency for Healthcare Research and Quality Pediatric Quality Indicators (PDI) measure sets are regularly utilized in pediatric hospitals to assess patient safety and quality and rank performance. No studies have examined concordance between them and the impact of selection on performance rankings.

Objective: This study evaluated whether choice of HAC or PDI measure set matters when evaluating hospital performance ranking.

Design/Methods: This retrospective cohort study used inpatient data from 42 hospitals in the Pediatric Health Information System (PHIS) for pediatric patients (0-17 years) discharged in 2021-2022. We extracted 8 HAC and 7 PDI measures, ranking hospital performance on each measure. We calculated a summary measure reflecting average performance using the geometric mean of ranks across each measure set and examined concordance in rankings. We also examined concordance for the specific measures of vascular catheter-associated infections (VCAI) that were present on both measure sets.

Results: There was a moderate-to-strong linear association between hospital rankings using the HAC vs. PDI measure sets (r=0.63; p< 0.001) (Figure 1), with no significant difference in overall ranking between measure sets (median difference=-0.5; p=0.84) and moderate agreement in ranking quartile (Klw=0.46). Absolute differences in ranking were ≥10 for 14 (33%) hospitals, 5-9 for 12 (30%), and 1-4 for 14 (33%), while only 2 (5%) hospitals were ranked identically on both measure sets. For indicators of VCAI, there was a strong linear association between both measure sets (r=0.91; p< 0.001) (Figure 2), with no significant difference in rankings (median difference=0.0, p=0.91) and substantial agreement in ranking quartile (Klw=0.79).

Conclusion(s): Summary rankings were broadly similar between HAC and PDI measure sets, but not identical. Some hospitals exhibited large discrepancies, ranking among the top performers on one measure set but near the bottom on the other. Selecting the appropriate measure set is important and can have implications for benchmarking, hospital reputation, and reimbursement. Pediatric hospitals should consider measure definitions and applicability to hospital-specific patient populations when selecting safety and quality measure sets for performance evaluation and ranking.

Table 1. CMS HAC and AHRQ PDI measures evaluated.
Table 1.pdfAbbreviations: Centers for Medicare and Medicaid Services (CMS); Hospital Acquired Conditions (HAC); Agency for Healthcare Research and Quality (AHRQ); Pediatric Quality Indicators (PDI); surgical site infection (SSI)

Figure 1. Overall ranking on HAC and PDI measure sets by hospital (N=42 hospitals).


Figure 2. Ranking for vascular catheter infections using HAC and PDI measures by hospital (N=42 hospitals).