External Review Letters in Promotion and Tenure Decision Making: Validity and Fairness

University Of Houston, Houston TX

Investigators

Abstract

Promotion and tenure decisions at universities are of critical significance to the integrity of the research enterprise because they determine the career progression of scholars and scientists. This study includes novel ways to examine gatekeeping mechanisms to academic careers such as the promotion and tenure (P&T). Despite the widely recognized importance of P&T processes, minimal research has examined whether and how P&T processes' most critical elements contribute to outcomes for STEM faculty. This project examines the validity and fairness of external review letters (ERL) provided by arm's length reviewers as part of promotion and tenure decision making. Academic administrators often view ERLs as the most impartial and critical components of tenure and promotion portfolios, providing qualitative, independent evaluations of candidates' past accomplishments, reputation, potential, and the prospect of continued, sustainable contribution levels. Despite their sensitive nature, given the criticality of ERLs in academic promotion and tenure decisions, the limited research on validity and reliability in ERLs is stunning. This project closes this gap by analyzing the linguistic characteristics of ERLs and examining the relationship between ERL linguistic characteristics, promotion candidate characteristics, letter writer characteristics, and promotion and tenure voting outcomes at the department, college, and university-level committees. The project examines ERLs through theoretical work on the social psychology of language use, social comparison processes, and social role theory. The project's goals are to investigate the validity of ERLs, determine possible reliability issues in ERLs based on candidate and letter writer characteristics, and identify possible ways to remediate limited validity and reliability. This project leverages a partnership between nine universities (University of Houston, Hampton University, Worcester Polytechnic Institute, Lehigh University, University of Alabama, Louisiana State University, Rice University, Texas A&M University, and University of Texas Rio Grande Valley) to examine more than 9,000 ERLs. This project aims to contribute to fill a research gap on the use of external reference letters for decision making on academic career milestones for STEM faculty. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.