Exploring Differences Between Instructors' Exams and How These Differences Produce Scores that Could Inaccurately and Inequitably Represent Student Understanding

Arizona State University, Scottsdale AZ

Investigators

Abstract

In many STEM courses, students' exam scores determine their course grades. In turn, when averaged together, course grades determine each student's grade point average, which can affect their persistence in a STEM major and their competitiveness for admission to professional or graduate schools. Thus, it is important that these exams accurately and equitably measure students' understanding of the subject matter that they are supposed to test. Little research has been done to determine whether specific exam questions accurately measure student understanding or whether they are fair to all students. This collaborative project between Arizona State University and University of Washington will try to fill this gap in knowledge by analyzing questions on introductory biology course exams taught by different instructors. They will examine the relationships between different types of questions, student scores on the questions, and student understanding of the concept that the question is supposed to test. This information has the potential to help biology instructors more fairly and accurately test student understanding of biology. It may also provide guidelines for building fair and accurate exam questions that are relevant to other STEM disciplines. This project has four key goals: (1) characterizing the composition of instructor-generated biology exams across sections of the same introductory biology course; (2) characterizing elements of questions that result in students performing differently on questions; (3) determining if modifying questions can diminish differences in student performance; and (4) correlating exam scores to students' conceptual understanding of biology. This project will focus on the exams of different instructors teaching the same introductory biology course offered across multiple institutions within the same regional network. No published studies have explored differences in question performance on instructor-generated exams in introductory biology courses. Further, this project would be the largest, most comprehensive analysis of instructor-generated exams and questions in any STEM discipline done so far, providing insights into what factors may affect performance differences on individual questions. This project can help instructors and researchers become more aware of elements of questions that result in unfair evaluation of certain groups of students, leading to more accurate and equitable measurement of student understanding.