310 - Offline Reinforcement-Learning Agent for Pediatric Sepsis Triage and Early Action Using Synthetic ED EHR

Publication Number: 1294.310

Miantong Zhang, Duke University, School of Medicine, Guangzhou, Guangdong, China (People's Republic)

Background: Timely antibiotic administration is critical for pediatric sepsis management, as each hour of delay increases mortality by ≥ 4%. Traditional triage rules and clinician judgment often lack adaptability to complex emergency department (ED) data. We aim to develop an AI triage agent that identifies pediatric ED patients at risk of sepsis and recommends the first time-critical action (e.g., initiate antibiotics). To ensure privacy and rapid experimentation, all training and evaluation were conducted on validated synthetic pediatric ED EHR that mirror real-world statistics without accessing protected health information.

Objective: To demonstrate the feasibility of learning a reliable pediatric ED triage policy from synthetic EHR that (i) detects likely sepsis early and (ii) recommends urgent next actions that may reduce antibiotic delays.

Design/Methods: Each encounter was modeled as 48 hourly windows. At each step, the agent received a 256-dimensional state derived from 29 vitals/labs, 194 medications, and six recent actions via a shared Transformer-GRU encoder. The policy acted over 10 discrete ED actions (e.g., start antibiotics, give IV fluids, escalate to ICU). An offline reinforcement learning policy using Conservative Q-Learning (CQL) was co-trained with a five-level acuity classifier on the shared encoder. Rewards prioritized earlier correct antibiotics, penalized delays and unnecessary use, and discouraged missed sepsis. All data were synthetic and IRB-exempt.

Results: CQL converged after ≈ 90 k gradient steps with fitted-Q return V = 0.141 ± 0.003 and final TD loss 0.0033 ± 0.0002, indicating stable value estimates and learning dynamics. The model achieved an action-matching rate of 65% relative to expert decisions, suggesting clinically aligned recommendations. End-to-end preprocessing, training, and evaluation completed within two hours on a single NVIDIA RTX 5000 Ada GPU. The learned policy consistently initiated antibiotics earlier for high-risk trajectories while avoiding over-prescription in low-risk cases.

Conclusion(s): Using validated synthetic pediatric ED EHR, an offline RL agent can effectively learn to identify likely sepsis and recommend urgent next actions-without accessing real patient records. These findings establish both feasibility and safety for future validation with de-identified or live clinical data, with the potential to reduce antibiotic delays and improve early pediatric sepsis care.

Figure 1. Offline Reinforcement Learning Agent Pipeline for Pediatric ED Sepsis Triage
The overall model workflow for developing a pediatric emergency department (ED) triage agent trained on validated synthetic EHR. The pipeline includes data preprocessing, feature encoding (vitals, medications, and action history), shared Transformer-GRU encoder, and offline reinforcement learning with Conservative Q-Learning (CQL). The agent learns to identify likely sepsis cases and recommend timely antibiotic initiation while preserving data privacy.

Figure 2. Comparison of Observed vs. Agent Policy Action Distributions in the Synthetic Pediatric ED Cohort
Comparison between clinician-observed emergency department (ED) actions and the offline reinforcement learning (RL) agent's policy after training on validated synthetic pediatric ED EHR. The agent reproduces major clinical patterns while selectively increasing early antibiotic initiation and ICU escalation, achieving an overall action-matching rate of 65%.