Collaborative Research: Bayesian Estimation of Restricted Latent Class Models

Board Of Regents, Nshe, Obo University Of Nevada, Reno, Reno NV

Investigators

Abstract

This research project will advance statistical methods known as cognitive diagnosis models (CDMs). CDMs are the statistical machinery that link cognitive theory with applications in online learning technology. They serve as a framework for providing fine-grained classification of the skills and attributes needed for success in the classroom and beyond. Robust cognitive theory is central to ensure accurate inferences with CDMs. This project will develop new statistical methods for validating cognitive theory in the context of CDMs. The modern classroom generates a wealth of longitudinal information from computerized student assessments. The innovations from this project will provide a framework for human development that harvests the available information to track skill development and to support teachers' instructional decisions in real time. The new methods will be applicable more broadly to other disciplines, such as the social sciences, neuroscience, medicine, and business, as an approach to gain more detailed and nuanced information about cognitive processes underlying human judgment and decision making. Software developed during the course of the project to implement the developed procedures will be made publicly available. The project will advance statistical and psychometric theory by developing Bayesian methods for estimating the Q matrix for a general class of models. Cognitive theory will be incorporated into CDMs by specifying a Q matrix that catalogues the skills required by each task. The general unavailability of cognitive theory Q matrices for most content areas and research domains poses a barrier to widespread application of CDMs. The project will offer several advances to existing research. The project will develop procedures for estimating Q for the most general restricted latent class model. Bayesian estimation methods will be employed that explicitly enforce identifiability conditions to ensure consistency and accuracy of parameter recovery. Stochastic processes and irreducible transitions will be created to estimate Q when the number of latent attributes is unknown a priori. The project also will consider methods that incorporate expert knowledge in the statistical model to improve estimation of Q and to enhance interpretation of uncovered attributes. Psychometric insights gained from working on this important problem can lead to significant developments in the underlying statistical theory for estimation of cognitive diagnosis model Q matrices and could potentially have an impact on the broad spectrum of applications beyond education and psychology, such as machine learning applications that seek to cluster binary data according to a set of underlying features. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.