The Advanced Placement Course Redesign Effort: A Time-Critical Analysis of Assessment Development Processes and Outcomes

University Of Illinois At Chicago, Chicago IL

Investigators

Abstract

This RAPID proposal builds on prior work to re-design the frameworks for AP science courses and proposes to capitalize on new sources of data that will be available from the Educational Testing Service (ETS) September 2009-August 2010. ETS is now piloting new assessments based on the re-design process. This project tests the quality of the assessment development processes that have been set in motion for creating new AP science exams. The new assessments follow a new evidence centered design approach (ECD) that is characterized by the alignment of the test items and tasks with the content domain models. The domain models for each course include: (1) a cognitive model of science practices, (2) a content outline, (3) achievement level descriptions, and (4) claims and evidence statements. The research will test how faithfully an ECD process can be executed with respect to the final phases of task and item development. The PI will conduct a review of the item specifications, review the draft biology and chemistry items, examine empirical data from the pilot tests, conduct interviews with the ETS and College Board staff members, and conduct cognitive laboratory protocols on the test items with students.