Collaborative Research: Improving the Effectiveness of Practice Exams in Large Introductory Physics Courses

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

This project aims to serve the national interest by improving students' success in introductory STEM courses. For many college and university students, "gateway" courses, such as introductory physics, present a roadblock to further study in STEM fields. Every semester approximately 4,000 students take introductory courses in the Department of Physics at the University of Illinois. Most of the students are engineering majors, because physics is a required course for all programs in the College of Engineering. Despite instructors' use of evidence-based instructional methods such as online pre-lectures, active-learning lectures, "just-in-time teaching," peer instruction, and collaborative problem-solving in recitation sections, nearly 15 percent of students do not pass the introductory courses. Most of these failing students actively participate in class and exert substantial effort, but nevertheless receive low grades on exams. When surveyed about the resources that they find most helpful in learning the material, students cite practice exams with video solutions as the highest-rated component of the course. Although considerable research shows that practice exams can be an effective form of studying, the degree of effectiveness depends on how the students engage with the exams. Indeed, simply reviewing problems and solutions develops familiarity with the material to be tested, but it does not often support the needed problem-solving ability. Consequently, lower-performing students become overconfident in their problem-solving ability and believe they are prepared to do well on exams, but instead they receive failing grades. This project will investigate whether changing the format of students' interactions with the practice exams, to encourage earlier use and to provide a more accurate assessment of understanding, will result in more effective studying behavior. If so, it may enable students to make a better self-assessment of their readiness and increase performance on exams. This project aims to advance knowledge in three areas that are critical for improving the effectiveness of practice exams as a learning resource, especially for underperforming students. First, it will explore students' difficulty in assessing their level of understanding, also known as the Dunning Kroeger effect, in STEM gateway courses. Although this effect has been verified in a variety of situations, the investigators will not only measure the size of the effect but also develop insights about whether improving students' ability to assess their level of understanding improves their performance (i.e., Is the correlation causal?). Second, the project will explore the role that implementation has on how students interact with practice exams and their effectiveness in helping students learn. In particular, the investigators will attempt to measure the impact that providing students a more realistic score on their practice exam has on the accuracy of their own understanding of how well they know the material. Third, the project will explore the impact that encouraging distributed practice can have on student performance, as well as the effectiveness of a relatively simple approach to encouraging that practice. In addition to guiding future education research, the knowledge gained from this project should maximize the effectiveness of practice exams for the at-risk students who need them most. At the University of Illinois alone, the results of the research should improve learning for thousands of students every year who hope to become engineers. However, the results should transfer to introductory physics courses at other institutions and introductory courses in other disciplines, thus magnifying the potential impact of this work. This collaborative project involves investigators at the University of Illinois at Urbana-Champaign (Award DUE-2021099) and Purdue University (Award DUE-2021190). This project is funded by the Improving Undergraduate STEM Education (IUSE: EHR) program, which supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.