Research Study of the LSAMP Program

Mathematica Policy Research Inc, Princeton NJ

Investigators

Abstract

The National Science Foundation (NSF) has a diverse portfolio of programs geared toward having broader societal impacts in terms of advancing discovery and knowledge by promoting education in science, technology, engineering, and mathematics (STEM); broadening participation of underrepresented groups in STEM; enhancing infrastructure for research and education through collaborations between institutions, industry, and others; and disseminating knowledge. The Louis Stokes Alliances for Minority Participation (LSAMP) Program and the LSAMP Bridge to the Doctorate (BD) fit perfectly within these goals, as do the activities proposed in this project. LSAMP aims to broaden STEM participation by building collaborations to increase the representation of minority students receiving STEM degrees and pursuing graduate study. The evidence and knowledge generated by this study will help NSF make programmatic decisions and should benefit LSAMP and similar programs. First, this study will help NSF determine the extent to which today's LSAMP Program aligns with current NSF priorities. Second, by comparing graduation data from the LSAMP monitoring data system and the National Student Clearinghouse (NSC), this project will gauge the quality and usefulness of estimates generated from data collected through the monitoring data system. This will help LSAMP provide guidance to grantees and improve its approach to data collection. Last, the subgroup analysis of LSAMP-BD will help inform NSF's decisions and build the knowledge base, given the lack of rigorous evidence on whether fellowships help increase the participation of underrepresented groups in STEM. The study's findings may also have an impact beyond NSF,particularly in agencies and foundations working to build and diversify the STEM workforce by providing evidence on the effectiveness of similar programs and guidance on using monitoring data systems and implementing reporting requirements for grantees. This study will expand on earlier evaluations of LSAMP and LSAMP-BD. It includes three distinct components. First, the LSAMP longitudinal data analysis will focus on the evolution and characteristics of the LSAMP Program over time, including activities offered and characteristics of the alliances, institutions, faculty, and students. This mostly descriptive and time-series analysis will draw on population data from the LSAMP monitoring data system. The second evaluation component will focus on verifying the quality of graduation data and the accuracy of these data among LSAMP Program participants. This will involve comparing graduation outcomes reported by institutions through the LSAMP monitoring data system with those reported to the NSC. This analysis will be based on a stratified random sample of LSAMP participants and will rely on standard t- and chi-squared tests to estimate significant differences between estimates generated from the two data sources. The third evaluation component, the BD subgroup impact analyses, will build on the recent BD evaluation by analyzing impacts by gender and prior preparation for graduate-level studies. Impacts will be estimated by measuring the (regression-adjusted) difference in average outcomes for the participant group (those receiving the BD fellowship) versus a matched comparison group (those not offered the fellowship and matched using propensity-score matching techniques). These impacts will be estimated using linear probability models to model the probability of reaching key milestones on the path to a Ph.D., including graduation.