Real-Time Multi-Dimensional Assessment of Student Learning

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

A web-based assessment instrument that provides real-time, multi-dimensional, formative assessment of student learning is being developed. This instrument provides measures of learning by topic and by level of mastery. The levels of mastery are defined as the information level, the algorithmic level, the conceptual level, and the problem solving level. The information level is characterized by memorization and the ability to recall, repeat pieces of information, and identify information that is relevant. The algorithmic level is characterized by the ability to mimic, implement instructions; and use memorized information in familiar contexts. The conceptual level is characterized by the ability to visualize, rephrase, change representations, make connections, and provide explanations. The problem-solving level is characterized by the ability to use material in new contexts; to analyze problems; to identify the information, algorithms, and understanding needed to solve them; to synthesize these components into a solution; and to evaluate the quality of this solution. Four products are resulting from the project: 1) A web-based assessment system founded on a coded database of questions; 2) A coded database of questions for use in measuring student learning in introductory college chemistry, calibrated to reproduce measurements generated through student interviews; 3) Protocols for developing and calibrating databases for use in other SMET courses; and, 4) Quantitative and qualitative analyses of the use of this system by both students and teachers, including the impact of the instrument on the users learning and teaching strategies and the impact of the instrument on student performance. Students are using the instrument to self-assess their learning after each assignment, to clarify expectations for their performance particularly with regard to conceptual understanding and problem solving, and to identify strategies for improving their achievement. Faculty are using it to prepare formative and summative assessments, to target particular practices and topics for improved instruction, to identify groups of students who need immediate assistance, and to assess the effectiveness of new teaching strategies and materials. Important co-funding of this project is being provided by the Division of Research, Evaluation and Communication.