ROLE Proposal: Study and Development of Distractor-Driven Multiple-Choice Tests Aligned to the National Science Standards

Harvard University, Cambridge MA

Investigators

Abstract

With a growing national facus on the measurement of learning outcomes in science classrooms, states and cities relying increasingly upon assessment methods developed prior to recent advances in cognitive scinece. While alternative and authentic assessments have gained a foothold in school reform, they are considerably more expensive and complex to utilize than traditional standardized tests that rely upon multiple-choice items. Rather than comdemn such simple item forms, the project plans to develop and study diagnostic assessment instrument that reflect advances in understanding conceptualchange in learner, yet are simple to administer and score. The prject will drawon the rich research literature documenting alternative conceptions and construct test items that allow choices between the scientifically correct answer to a problem and the most prevalent alternative conceptions. Earlier work shows that the psychometric profiles of such items depart significantly from those constructed and used by the makers of standardized tests, and that more research is needed before relucant test-makers will adopt items with profiles that depart from their non-cognitive models. Such a study is imperative for the development of standarzed tests that accurately measure progress toward achievement of swtate and national science standards. In addition, analytical methods will be developed and employed to reveal the underlying prerequisite structure of such tests, shedding light on the proper sequence of ideas in K-12 instruction. Results will be publicized through academic publications and papers. A public domain database of diagnostic items will be made available for test developers and educators.