A Regression Discontinuity Study of the Impact of Mathematics Achievement Fund on Student Outcomes

American Institutes For Research In The Behavioral Sciences, Arlington VA

Investigators

Abstract

Socioeconomic and racial gaps in STEM (science, technology, engineering, and mathematics) participation can sometimes be traced back to discrepancies in mathematics skills that emerge at kindergarten entry. Importantly, children who exhibit early lags in mathematics development may also demonstrate lower growth rates in mathematics skills later. Consequently, early gaps in mathematics skills may widen over the course of schooling. How to diagnose and correct mathematics shortcomings early is a challenge that K–12 schools must confront systematically. Unlike reading, few states currently have a coherent statewide program for early mathematics intervention. Kentucky’s Mathematics Achievement Fund (MAF)—a statewide K-3 mathematics intervention program—is a prime example of a state’s effort to intervene early on to change a student’s mathematics development trajectory. By using a quasi-experimental design with data gathered during the implementation of MAF, this study will provide valuable information on how states can structure and provide resources to support early mathematics intervention efforts. The project will connect research and practice and add much-needed information to the existing knowledge base that states can use when considering early-grade mathematics programs. The proposed project is a joint effort between the American Institutes for Research and the Kentucky Department of Education and is supported by the EHR Core Research (ECR) program, which supports work that advances fundamental research on STEM learning and learning environments, broadening participation in STEM, and STEM workforce development. Existing early-grade mathematics interventions tend to be one-off, localized efforts, usually without a coherent or sustainable strategy, and often implemented by experimenters or supported by vendors of prepackaged interventions. Interventions that show promising results are typically small-scale studies, whereas large-scale studies often find null benefits. This project is designed to examine the causal effects of MAF on student outcomes in a regression discontinuity framework. In particular, the project will use student-level administrative data and original data collected through surveys and interviews to examine the causal effects of MAF on students’ academic and non-academic outcomes both immediately after the intervention and in later grades. The project will also investigate who is likely to benefit from MAF and under what conditions and conduct a cost-effectiveness analysis of the program. The project will contribute to the existing literature in several ways. First, most existing studies on early mathematics interventions focus on student outcomes at the end of the intervention. There is evidence that large, short-run effects on student scores in early grades could dissipate over time. By tracking some 3rd-grade students into high school, the project plans to examine the extent to which the short-term benefits of the MAF program, if any, persist over time. Further, the project will also explore how MAF may affect other aspects of children’s schooling experiences such as possible spillover effects of MAF on other subjects and non-test outcomes like middle school course-taking and student engagement. In this way, the proposed project seeks to catalyze new knowledge that will help inform the discussion on systematic, coherent early-grade mathematics policies at the state level. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.