Emerging Research-Empirical: Development and Application of a Multilevel Multiple-Group CDM to Compare Cognitive Attribute Distributions based on Eighth Grade TIMSS Mathematics

Teachers College, Columbia University, New York NY

Investigators

Abstract

The proposed research project will develop new extensions of cognitive diagnostic models (CDM) to understand what math skills U.S. students possess or lack, and compare the distributions of skills across countries and within countries across years. The research team will examine the relationship between these skills and core background variables (e.g., gender). The research focuses on the TIMSS eighth grade mathematics assessments from 1999, 2003, and 2007. To handle the complex sampling design used in TIMSS, the PIs will develop the multi-group CDM to account for the clustering of students within schools. Whereas the TIMSS framework identifies only a single skill for each item (e.g., ability to calculate area) the proposed research will identify multiple skills for each item. This qualitative analysis is required as part of the CDM, but is also important to better understand the TIMSS assessments. The statistical methods developed as part of this research will allow a thorough analysis of the TIMSS assessment data. As part of the proposed research, the principal investigators will work with math education researchers and measurement experts to develop statistical models and codes to analyze assessment data. Although the proposed research focuses on eighth grade TIMSS mathematics data, the statitical methods will have implications for use with other national and international data and assessments (e.g., NAEP and PISA). Policy makers will be able to use this information to make well-informed decisions about curriculum changes. Future studies will be able to use the methods to examine different age groups (e.g., 4th grade), different subject areas (e.g., science), and different assessments (e.g., NAEP and PISA).