363 - AI-Enabled Auto Capture of Throat Images for Smartphone-Based Oropharyngeal Examination in Pediatric Patients

Publication Number: 2352.363

Youness Arjoune, Children's National Health System, washington, DC, United States; Trong N. Nguyen, AusculTech Dx, Silver Spring, MD, United States; Rana F. Hamdy, Children's National Health System, Washington, DC, United States; Uday Santhosh Raju. Vysyaraju, AusculTech DX, Wheaton, MD, United States; Emily Ansusinha, Children's National Health System, Washington, DC, United States; Patrick Dolan, PM Pediatric Care, Palatine, IL, United States; David Mathison, PM PEDIATRICS, Bethesda, MD, United States; Jeffrey S. Dome, Children's National Hospital, Washington, DC, United States; Raj shekhar, Children's National Health System, Washington, DC, United States

Background: Telehealth visits with medical providers are increasingly common and expand access to care, especially for patients in remote or rural areas. However, conducting a physical examination virtually presents challenges. In particular, examining the oropharynx-a key component of evaluating possible oropharyngeal infections-can be difficult in pediatric patients, who often struggle to keep their mouths open and positioned properly for an adequate view on camera.

Objective: The goal of this project is to develop an artificial intelligence (AI) technology that can assist parents and caregivers to capture diagnostic-quality throat images in home settings to augment a telehealth visit.

Design/Methods: We developed an auto-capture technology (Figure 1) that uses a trained convolution neural network (CNN) and runs in real time to detect relevant structures. When all desired structures are visible with good focus, the camera is triggered to capture a good image. To train and test this technology, we used a large database of patients enrolled prospectively as part of the parent study. The IRB-approved parent study includes patients ages 3 to 17 years with sore throat in whom an image of the oropharynx is taken by research assistants with a smartphone. Ground truth identifying the anatomic structures of the oropharynx on each image was identified by manual review of images by clinicians. To test the developed auto-capture technology, research assistants used manual capture and auto capture on the same patient (Figure 2). The captured images were scored for quality by the clinical research team members as good, marginal, and poor images (Figure 3).

Results: The CNN model trained on a data set of 6,347 throat images with ground truth (annotated structure boundaries) achieved a greater than 90% accuracy on all structures including the uvula, tonsils, pharynx, and palate, that combined together form the region of interest. Preliminary results demonstrated that the percentage of good images with auto capture was 35.9% vs 30.7% with manual capture, a 5.2% improvement.

Conclusion(s): The auto-capture feature can assist parents and caregivers in acquiring high-quality throat images to augment a telehealth visit.

Figure 1. A screenshot of the auto-capture feature.


Figure 2. Examples of auto-capture (left) and manual images (right) of the same patient.


Figure 3. Examples of good-, marginal-, and poor-quality throat images.