271 - Cheating the Baseline: Exploring Reliability and Manipulation in Remote Baseline Concussion Testing

Publication Number: 2261.271

Joseph Mekhail, Northwell Health, New York, NY, United States; Melanie Malkasyan, Northwell Health, Fair Lawn, NJ, United States; Caroline Howard, Northwell Health, New York, NY, United States; Lillian M. Ravikoff, Northwell Health, Bronx, NY, United States; Ruth Milanaik, Northwell Health, Great Neck, NY, United States

Background: Sports-related head injuries are a leading cause of mild traumatic brain injury (mTBI) in youth athletes. Baseline concussion tests (BCTs) are administered pre-season and repeated after a head injury to detect neurocognitive changes that may indicate impairment. HitCheck (HC) is a free mobile app that uses game-like tasks to assess cognitive performance pre- and post-injury. However, studies have reported that some athletes intentionally underperform at baseline, a practice known as "sandbagging," to manipulate post-injury comparisons and influence return-to-play decisions. Although digital concussion testing tools such as HC have performance thresholds that prompt retesting when scores fall outside acceptable ranges, sandbagging may still occur because users can repeat BCTs until a desired baseline is recorded.

Objective: To assess the potential for intentional underperformance with remote BCTs.

Design/Methods: A youth research team (n=14, mean age=20, F=6) completed the HC BCT under 2 conditions: performing to the best of their ability (pre-test) and intentionally underperforming to lower their baseline scores without detection (post-test). HC evaluates 9 domains of neurological function: Immediate Recall, Balance, Reaction Time, Coordination, Pattern Recognition, Problem Solving, Color Recognition, Impulse Control, and Delayed Recall (Figure 1). 9 repeated-measures ANOVAs were conducted to compare pre- and post-test scores across each cognitive domain in HC. A two-sample t-test compared mean total score between conditions.

Results: Participants were able to lower their overall scores from their genuine baseline (p=.006)(Figure 2). Significant changes in 7/9 HC domains were possible: Immediate Recall (p=.006), Balance (p=.005), Coordination (p=.04), Pattern Recognition (p=.005), Problem Solving (p=.008), Impulse Control (p=.04), and Delayed Recall (p <.001). Delayed Recall and Pattern Recognition had the largest performance declines, indicating greater susceptibility to user manipulation. Color Recognition and Reaction Time showed no significant differences, suggesting these domains may be harder to manipulate.

Conclusion(s): Participants could intentionally deflate BCT scores, highlighting concerns regarding the reliability of remote BCT and its role in diagnosing mTBI. Baseline assessments should be administered under adult supervision to maximize BCT accuracy. Further research with a larger sample of youth athletes is needed to validate these findings and guide the development of underperformance detection methods. A medical professional should be consulted regardless of results after head injury occurs.

Figure 1. HitCheck's Game-Like Tasks to Assess Cognitive Performance
Eight domains of cognitive assessment from HitCheck: (1) Immediate Recall, (2) Balance, (3) Reaction Time, (4) Coordination, (5) Pattern Recognition, (6) Problem Solving, (7) Color Recognition, and (8) Impulse Control. Note: the ninth domain (Delayed Recall) is not included in the panel as the interface is the same as (1) Immediate Recall.

Figure 2. Mean Total BCT Scores by Testing Condition
Bar plot indicates that participants significantly lowered their overall scores when intentionally underperforming (mean = 74.2) compared to genuine baseline performance (mean = 92.0) (p=.006). Mean overall BCT score was calculated for pre and post-test conditions. The highest score was 100 for all domains except for Reaction Time which was in seconds; mean Reaction Time for both conditions was normalized to 100 using (1 - ReactionTime) * 100).