Score-Based Tests of Measurement Bias in Explanatory Item Response Models

University Of Missouri-Columbia, Columbia MO

Investigators

Abstract

Large-scale, standardized ability tests have a prominent role in society, impacting college admissions, occupational decisions, and resource distribution. This research project will involve the development of novel statistical methods that make it easier to determine whether or not the items making up a test are fair. If a test (or specific items on a test) were to unfairly advantage one group of students over others, then some students from the disadvantaged groups may miss out on the opportunities that they deserve based on their abilities. Thus, it is important to ensure that standardized tests are fair to the diverse groups of students taking the tests. The new methods will be implemented in free software, and all data resulting from the project will be openly disseminated via the internet. The project also will support a doctoral student in psychometrics, which is a STEM discipline with a documented shortage of students. This project will develop statistical methods that are designed to be used with item response models. Item response models are considered to be state-of-the-art methodology for educational test development and analysis. The new methods involve generalizations of the score test (also known as the Lagrange multiplier test) that is well known to statisticians. These generalizations have been relatively unexplored in psychometrics, so that the methods developed in this project will address unresolved problems related to the study of test fairness. In particular, these new methods will allow one to study novel hypotheses of fairness using simpler statistical models than are required for traditional methods. The new methods will further allow for the study of fairness across many groups of students and within a large class of item response models, many of which are more complex than traditional models. Along with theoretical development and software implementation, the project will illustrate the methods' abilities via simulation. The simulations will directly compare the new methods to traditional methods and compare multiple novel statistics to one another. The project will provide researchers with novel methods to study test fairness, free software to carry out the methods, and simulation results that guide researchers in the methods' optimal uses.