Helping Improve and Scale Introductory Programming Courses through Automated Code-Reading Exercises

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

This project aims to serve the national interest by developing a tool that will help computing departments to prepare more students to enter the technical workforce, in particular to improve their ability to program. Explain in plain English" (EipE) questions are considered by many to play an important role in helping students to learn to program by emphasizing “code reading,” an important developmental skill that supports “code writing.” However, while many programming course activities (e.g., code writing) can be objectively graded in a straight-forward manner, activities like "Explain in plain English" (EipE) questions that ask students to read a given piece of code and describe its function in English are difficult to grade consistently. While EipE questions are well regarded by researchers, they are not in widespread use instructionally, presumably due to the burden of manually grading them and the slow feedback provided to students. The goal of this project is to develop a natural language processing (NLP) based autograder for EipE questions that will provide students with immediate feedback, ease the instructional burden of using EPiE questions in the classroom, and help increase the use of this effective pedagogical approach. Starting from a bag-of-words and bigram-based implementation that is already accurate enough for use in low stakes assessments, this project will refine this implementation using publicly available pre trained Transformer architectures. The project centers around two research questions: (1) How does student performance on code reading activities relate to performance on other activities in introductory programming courses? and (2) Can we improve student success rates in introductory programming courses by introducing automated formative EipE assessments? The proposed qualitative and quantitative studies will contribute to the understanding of how novices learn to program and the role that learning to read code plays in learning. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.