Faster Mixing Markov Chain Monte Carlo for Multidimensional IRT and Cognitive Diagnosis Models

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Fitting complex item-response theory (IRT) models has become increasingly important in educational assessment, psychological assessment, and patient-reported health assessment. The use of Markov Chain Monte Carlo (MCMC) methods for fitting these models has surged in recent years, particularly for multidimensional IRT and cognitive diagnosis models where standard methods such as maximum likelihood estimation are difficult to apply. Unfortunately, slow mixing of the Markov chains is often a severe problem, limiting the ability of researchers to draw valid inferences from the MCMC output. This project will develop fast-mixing MCMC algorithms applied to complex IRT models including multidimensional IRT models and cognitive diagnosis/diagnostic classification models. Specifically, the improved algorithms will include rescaling and re-centering approaches such as Meng and van Dyk's conditional and marginal augmentation, methods using gradient information, and adaptive MCMC methods. The project will extend the analytical reach of advanced MCMC methods in the psychometric literature, focus the attention of the psychometric community on MCMC mixing, and provide the basis for effective MCMC implementation in complex models. Efficient and accurate fitting of IRT models is of profound importance. The high-stakes nature of most educational assessments demands valid estimation procedures. Therefore, slow mixing, leading to either slow or non-convergence of MCMC estimation procedures, can profoundly affect inference, leading to incorrect ranking of individuals. The methods developed by this project will address a critical problem in applications of MCMC in the psychometric literature. The project also will develop free, open-source software with which to implement the new methods.