Advances in Local Independence and Unidimensionality Assessment for Dichotomous and Polytomous Item Exams

University Of South Carolina At Columbia, Columbia SC

Investigators

Abstract

This research is concerned with the statistical analysis of large-scale standardized educational and psychological tests. The data from these tests are commonly analyzed using the item response theory (IRT) paradigm. Many IRT based procedures rely on the assumptions that testing data can be modeled as if it measures a single ability or dimension (unidimensionality), and that the responses to the questions on the exam are independent of one another once the examinee's ability level is known (local independence). The standard procedures for verifying that these two assumptions hold suffer from a variety of weaknesses. In particular, they either require additional assumptions about an underlying parametric statistical model or an ad-hoc correction for statistical bias. This research will refine recently proposed methods for verifying these assumptions by using the parametric bootstrap procedure and nonparametric regression. The research will expand the theory of the parametric bootstrap and nonparametric regression from other areas of application to IRT. This will be done both for tests whose items are scored simply as correct or incorrect as well as for those that allow for partial credit. The end result of this research will be a theoretically justified, computationally feasible method of verifying the assumptions of unidimensionality and local independence without need of a specified parametric model or an ad-hoc bias correction. The role of standardized tests has been ever increasing in education in the United States. They are used for everything from determining college admissions decisions to attempting to judge the effectiveness of schools and school districts. As such, it is vital to continually improve the statistical methods used to insure the accuracy and fairness of these standardized tests. Many of the statistical procedures currently used to insure the reliability of these tests are based on some underlying technical assumptions that can be difficult to verify. The result of this research will be methods that the practitioner can use to insure that the procedures for verifying the accuracy and fairness of the standardized tests are performing as expected. Given the historically close interaction and spirit of collaboration between the large national testing companies, university researchers, and state and local departments of education, these results will be immediately disseminated to agencies which directly affect every student in America. In addition to this immediate application, this research also will provide insights into the future development of reliable methods for insuring accuracy and fairness of standardized tests even when the desired underlying technical assumptions are not met.