267 - Gender and Racial Differences in Letters of Recommendation for Residency Candidates

Publication Number: 1253.267

Colin M.. Sox, Boston University School of Medicine, Newtonville, MA, United States; Elizabeth Johnson, Harvard Business School, Cambridge, MA, United States; Bushra Guenoun, Harvard Business School, Cambridge, MA, United States; Robin J. Ely, Harvard Business School, Boston, MA, United States; Sharon Reale, Brigham and Women's Hospital, Boston, MA, United States

Background: Whether disparities exist in residency candidates’ letters of recommendation is unknown.

Objective: We sought to compare letter length and language by race or gender.

Design/Methods: We conducted a retrospective cohort study of applicants from United States (US) medical schools to residency programs in pediatrics, anesthesiology, emergency medicine, internal medicine, and surgery at a tertiary care center in Boston, MA. Using natural language processing, we analyzed applications’ letters of recommendation to determine word count and levels of warmth, sociability, concreteness, and morality. Our primary predictors were applicant gender and race under-represented in medicine (“URIM”). Our main outcome was invitation for residency interview. We conducted multivariate logistic and linear regressions to control for applicants’ academic performance, medical school rank, and applicant-, author-, and letter- characteristics.

Results: Of 6,817 subjects, 1,349 self-identified as URiM (20.7%), and 1,164 were invited to interview (17.1%). The mean word count of 22,142 letters of recommendations was 503.4 (SD: 305.1). Letters supporting URIM applicants were 37.2 words shorter than non-URMs’ letters, a significant difference (Adjusted Standardized Mean Difference [aSMD]: -0.06; 95% Confidence Interval [95% CI]: -0.11, -0.01). Letter length didn’t significantly differ by gender (aSMD -0.01; 95% CI: -0.05, 0.03). Higher word count was significantly associated with interview invitation (Adjusted Odds Ratio [aOR] = 1.17, 95% CI: 1.09 – 1.26).

Compared to non-URiM applicants, the language in URIMs’ letters scored significantly higher in Sentiment (aSMD: 0.08, 95% CI: 0.02 - 0.13), sociability (aSMD: 0.14, 95% CI: 0.08 - 0.20), and morality (aSMD: 0.11, 95% CI: 0.05 - 0.17). Compared to males, females’ letters scored lower in concreteness (aSMD = -0.14, 95% CI: -0.17, -0.08) and higher in assertiveness (aSMD: 0.07, 95% CI: 0.03, 0.11). URIM applicants were significantly more likely that non-URiMs to be invited to interview (aOR: 3.05; 95%CI: 2.32, 3.99); in contrast, interview invitations did not significantly differ by gender (aOR: 1.14; 95%CI: 0.93, 1.40).

Conclusion(s): Using natural language processing to analyze letters of recommendations from applications to residency in five specialties, we identified racial and gender disparities in letter length and fundamental dimensions of intergroup perception. Race and gender were also associated with subsequent interview invitations.