751 - “TEACH”ing Advocacy: Assessment of trainee advocacy messages with an evidence-based assessment tool

Publication Number: 3727.751

Jessica Weisz, Children's National Health System, Washington, DC, United States; Greer Eckard, George Washington University School of Medicine and Health Sciences, Washington, DC, United States; Tamara Gayle, Children's National Hospital, Washington, DC, United States; Aarenee Greene, Children's National Health System, Reston, VA, United States; Shaynel Philpotts, Children's National Hospital, Rockville, MD, United States; Olanrewaju Falusi, Children's National Hospital, Washington, DC, United States

Background: Creating effective advocacy messages is an important skill for trainees. However, the literature lacks standardized tools for assessing trainee advocacy messaging.

Objective: To analyze trainee ability to craft effective advocacy messaging with a new assessment tool.

Design/Methods: Residents participated in the Policy and Child Poverty component of the Trainee Education in Advocacy and Community Health (TEACH) Curriculum, a multimodal child poverty and advocacy curriculum developed at our institution and available online for any training program. Trainees completed a module on legislative processes and advocacy messaging then attended or virtually viewed a federal or local legislative hearing. Afterwards, trainees were tasked with producing an effective advocacy message. Medical educators collaborated with public health professionals and governmental relations staff to develop a tool to assess residents' advocacy messages. After a review of available literature and online resources, the American Academy of Pediatrics Advocacy Guide "Tips for Writing 'Printable' Letters to the Editor" served as the foundational framework. A scoring rubric was iteratively designed with Likert-scale questions for 5 domains and two free-text questions on strengths and areas for improvement. The study team trained 8 faculty members to use the rubric; each assignment was scored independently by a faculty member and a graduate student. Mean scores for each domain were calculated. Common themes across advocacy messages were identified. Free text responses were reviewed for themes. Trainees rated their satisfaction and confidence in a post-evaluation.

Results: 73 submissions were reviewed. The most frequent advocacy topics were youth mental health (n=14), education (n=11) and cost of childcare (n=9). The highest scored domain was use of common language and lowest was counteracting the counterargument (Table 1). Free-text comments commonly cited "passion" in the resident's writing as a strength as well as use of evidence and ability to explain complex concepts that impact health outcomes. For constructive comments, common themes were the need for a more specific call to action, a more relevant patient story, and a more thorough counterargument. Residents responded positively to the experience and were more confident in their skills after participation (Table 2).

Conclusion(s): Development of effective messaging skills is a key component of advocacy training. This tool is unique and offers educators a model to give structured feedback to learners. Next steps include collecting more validity evidence for the tool.

Table 1: Resident Scores from Advocacy Message Assessments


Table 2: Post-Test Evaluation