Doctoral Dissertation Research: Hierarchical Item Response Models for Cognitive Diagnosis

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

Cognitive diagnosis models have received increasing attention within educational and psychological measurement. The popularity of these models largely may be due to their perceived ability to provide useful information concerning both examinees and test items. However, the validity of such information may be undermined when diagnostic models are misspecified. This project focuses on one aspect of model misspecification: violations of the local item independence assumption. It examines potential causes and consequences of such dependence, with particular attention to those causes unrelated to the attributes a diagnostic test is intended to measure. The project proposes and evaluates a hierarchical diagnosis model as an alternative to traditional diagnosis models in which nuisance dependence is ignored. This model maintains the desirable properties of existing models while allowing for greater complexity in the underlying response process. Importantly, the model may be estimated efficiently, even for models with a large number of nuisance latent variables, using an analytical dimension reduction technique described by Gibbons and Hedeker (1992). There is growing interest in extracting model-based diagnostic information from assessments in order to provide more useful feedback to stakeholders. Up to this point, however, the question of whether traditional cognitive diagnosis models fit real test data has been somewhat neglected. This project examines the issue of model fit and presents a model that may better account for certain causes of misfit than the traditional diagnosis models. To the extent that the proposed framework better accounts for the structure of real test data, its application will contribute to improved test development and lead to more valid model-based diagnostic inferences, such as the classification of test takers according to cognitive attributes or skills. This, in turn, is expected to enhance decision making, as better diagnostic information may allow for more effective (or better targeted) delivery of instructional strategies or clinical interventions. The results from this project will benefit the psychometric practice in any social and behavior science discipline that involves testing and measurement. As a Doctoral Dissertation Research Improvement award, support is provided to enable a promising student to establish a strong, independent research career.