153 - Derivation of Scenario-Based Simulation Checklists to Identify Serious Patient Safety Events in Common Prehospital Pediatric Emergencies

Publication Number: 4150.153

Tehnaz Boyle, Boston Medical Center, Boston, MA, United States; Janette Baird, The Warren Alpert Medical School of Brown University, St John, Virgin Islands, United States; Kathleen Adelgais, University of Colorado School of Medicine, Aurora, CO, United States; Marc Auerbach, Yale School of Medicine, New Haven, CT, United States; Mark Cicero, Yale School of Medicine, New Haven, CT, United States; Richard L.. Lammers, Western Michigan University Homer Stryker M.D. School of Medicine, Mattawan, MI, United States; Ijeoma Okafor, Boston Medical Center, Boston, MA, United States; Robert H. Seitter, Boston University School of Medicine, Brookline, MA, United States

Background: We need experimental models to test how patient safety interventions impact Emergency Medical Service (EMS) care of children. Previous simulation models identified errors by the absence of optimal performance rather than the presence of actions causing harm. This approach lacks sensitivity and excludes practice variation limiting generalizability. Tools to estimate and grade harm arising from errors of omission and commission are needed to optimize EMS care.

Objective: To derive scenario-based checklists that identify prehospital errors causing moderate-to-severe harm or death (serious safety events, SSE).

Design/Methods: This modified Delphi study used expert consensus and survey methods. First, 4 pediatric emergency care experts adapted validated simulation performance checklists for prehospital scenarios (cardiac arrest, asthma, seizure, anaphylaxis) to derive initial SSE checklists. The team used research, guidelines, and expert opinion to prioritize observable SSEs, add/modify thresholds to include practice variation, and remove items causing minimal-to-no harm. Next, we used purposive sampling to recruit EMS medical directors, patient safety and quality assurance personnel from national groups as Delphi experts. Participation was voluntary and remunerated. Delphi experts independently evaluated checklist items in asynchronous web surveys in iterative rounds until all items reached consensus (≥80% agreement). Case-dependent items unique to a scenario (e.g., CPR in cardiac arrest) and case-independent items occurring in >1 scenario (e.g., basic airway management) were evaluated once per round. An item was included if ≥80% experts agreed it caused moderate-to-severe harm or death, excluded if ≥80% agreed it caused minimal-to-no harm, and revised by expert feedback if neither criterion was met.

Results: Of 56 invited Delphi experts, 16 consented and 13 completed all surveys (81% response). Delphi experts represented 11 states (Table 1). After adapting the initial checklists and excluding 73 duplicate items, Delphi experts evaluated 86 unique items (Table 2). After 4 rounds, 77 items met consensus for inclusion in the final SSE checklists. Table 3 summarizes SSE checklist items by care domain.

Conclusion(s): We derived new prehospital patient safety checklists to identify clinically important safety events by observing simulated pediatric EMS encounters. In future work, we will test tool reliability and psychometric properties in simulated and real patient encounters before use in research and education.

Table 1. Participant characteristics.


Table 2. Comparison of the total number of items for each prehospital emergency scenario in the original performance checklist, the initial SSE checklist after adaptation (pre-Delphi), and the final SSE checklist (post-Delphi).
*Counts include duplicate items occurring in >1 case related to basic and advanced airway management, vascular access, drug and fluid therapy.

Table 3. Number of items in each care domain included in the final SSE checklist for each prehospital scenario. Case-dependent items were unique to the scenario while case-independent items occurred in more than one scenario.
Abbreviations: CPR = cardiopulmonary resuscitation