Collaborative Research: Assessing the Longitudinal Impact of Noyce Awards on the Subject Matter Knowledge of Beginning STEM Teachers in the US: A Comparative Study

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

In the 2010 National Research Council's report, "Preparing Teachers: Building Evidence for Sound Policy," areas related to STEM teacher preparation that lacked a sufficient research base were highlighted. This Noyce Track 4B research project seeks to address one of the areas identified as it studies the extent to which those institutions receiving Robert Noyce Teacher Scholarship (Noyce) funding from the National Science Foundation have prepared more knowledgeable beginning teachers than those institutions which have not received Noyce grants during Noyce funding cycles as measured by outcomes on Praxis II subject matter knowledge exams given to first time test takers. Through a collaborative research effort between Stony Brook University and Kennesaw State University, and working with the Educational Testing Service (ETS), the project will undertake the disaggregation of Praxis II test scores over the past fifteen years to determine to what extent those institutions receiving Noyce funding have prepared more knowledgeable beginning teachers than those without Noyce awards. The list of Noyce awardees from the NSF website will be used to categorize Praxis II test takers coming from Noyce and non-Noyce institutions over the past fifteen years. Using the self-reported demographic data from Praxis II test takers, the institutional codes will be compared to the Noyce and non-Noyce institution list and a simple count of students at each institution will be performed. As not all states have used Praxis II exams for licensure, only a subset of the Noyce portfolio will be captured in the analysis. The measure of teacher knowledge will be analyzed at two levels related to outcomes on the Praxis II subject matter knowledge exams. Level 1: analysis will use individual state cut scores for the respective year exams. This will provide a look at whether institutions are preparing STEM teacher candidates for a level pre-determined by their respective states. Level 2: analysis of Praxis II subject matter knowledge for a raw percent correct score on the exam. An average exam score will be calculated for each exam and teacher candidates will be placed into groups based on their performance relative to the mean (i.e., 1 standard deviation above the mean, 1 standard deviation below the mean, etc.). The outcome variables described in Level 1 and Level 2 will be analyzed as well as additional differences in performance by demographic variables for the total population and also by Noyce and non-Noyce institutions. Example questions that will be answered include: Do Noyce institutions prepare more minority STEM teacher candidates compared to non-Noyce institutions? Do women at Noyce institutions outperform women at non-Noyce institutions based on STEM Praxis II content knowledge exams? Do Noyce institutions have a higher number of non-traditional first-time Praxis II test takers compared to their non-Noyce peers? In order to develop a model for impact of Noyce on different institutions, average incoming SAT scores will be used as a covariate in the regression analysis that will control for selectivity of the institution. In the descriptive analysis, only schools with similar entrance requirements and average SAT/ACT scores to each other will be used. In order to measure the impact of the Noyce funding related to the quality of the scholars in situations where a university has had both Noyce scholars and non-Noyce scholars, data on the number of test takers in each Praxis II STEM field, their average GPA, and whether or not they have in-field majors prior to Noyce funding will be used to establish a baseline for number and type of teacher education graduates. This will allow for analysis of subsequent data to determine whether these patterns changed after receiving Noyce funding. An empirical study will be conducted limited to a subsample of the NOYCE schools to examine the effect of the proportion of Noyce graduates to total STEM education graduates in the given year. It will be included as a coefficient in the regression models to adjust the comparison of averages between the Noyce institutions based on the size of the proportion. If there is evidence that the ratio is related to the average PRAXIS scores at the institution level, then it will be collected on additional schools and incorporated into the comparison of averages between Noyce and non-Noyce institutions. Ultimately, this study will examine the influence of the Noyce program on the depth of content knowledge for beginning STEM teachers as they enter the profession.