Determining Algebra Readiness and Cognitive Skills Profiles by Using a Diagnostic Cognitive Model to Set a Performance Standard on a Pre-Algebra Test

Virginia Polytechnic Institute And State University, Blacksburg VA

Investigators

Abstract

The primary purpose of the research is to investigate the extent to which a performance standard can be set for an assessment instrument using a diagnostic cognitive model (DCM). The proposed Pathway research project is to use of an emerging psychometric methodology to investigate different approaches to establish a performance standard on an instrument for algebra readiness. The proposed research will use TIMSS algebra items and involve a secondary analysis of the 2003 TIMSS U.S. Grade 8 data. The research will focus on the skills and knowledge required for algebra readiness. This research project involves both quantitative and qualitative studies. The qualitative study provides strong evidence to support the findings of the quantitative investigations. The use of Diagnostic Cognitive Models to inform standard setting is exciting because it offers both a clearly defined methodology, but also because it promises diagnostic information regarding the strengths and weaknesses of each student so that teachers can design a specific remediation to meet each student's needs. To date there is no research that has examined the topic of setting performance standards using a diagnostic cognitive model (DCM). This research project offers a promising way forward for improving processes of standard setting, such that they rely less on the inscrutable judgments of panel members, and relatively more on factors that can be easily measured and related to test outcomes, such as DCM. The approach is potentially transformative not only for its potential to change how standard setting is conducted, but also for its potential to change the relationship between standards and teachers' ability to intervene effectively in the classroom, by targeting aspects of the DCM that best predict algebra readiness. The findings of the research project have broader implications for both instruction and policy. This research project cannot only advance our understanding of related issues in educational measurement and mathematics education, but also establish a model for two camps to communicate and work together for value-added research endeavor.