IUSE: EHR: Improving Undergraduate Algorithms Instructions with Online Feedback

Worcester Polytechnic Institute, Worcester MA

Investigators

Abstract

With support from the NSF Improving Undergraduate STEM Education Program: Education and Human Resources (IUSE: EHR), this project aims to serve the national interest by leveraging artificial intelligence (AI) techniques to improve student learning and instructor productivity in computing courses. An important aspect of an instructor's job is to provide feedback on student work. Research in cognitive science demonstrates that students more effectively learn concepts if they explain answers to cognitively demanding questions. Learning is further improved when instructor feedback is provided. In large classes, it is difficult for instructors provide all students with the feedback and help they need. This project will develop a tool that uses machine learning to automatically generate feedback on student solutions. The tool is designed to reduce the grading burden of the instructor while ensuring that students receive frequent and worthwhile feedback. The project aims to generate knowledge about how AI can help to engage students without requiring significant instructor time. This tool will impact thousands of students as it is incorporated into ASSISTments, which is used by 50,000 students and is a free service of Worcester Polytechnic Institute (WPI). The goal of this project is to implement and study a tool that uses state-of-the-art machine learning techniques and natural language processing methods to automate instructor feedback. In the first phase of the project, students in the Algorithms course at WPI will be assigned open-ended questions to complete prior to each lecture. Teaching staff for the course will manually grade student responses. The data set generated in this phase will be used to develop and train the algorithms for the automated tool. The tool will apply methods of varying complexities including regression and tree-based modeling methods, and deep learning methods including long short-term memory (LSTM). It will analyze student responses and suggest feedback and a grade. Instructors may choose to accept the tool's suggestions or override the suggestions to provide different feedback. A randomized control trial will assign students to receive manual feedback or feedback generated with the help of the tool. The usability of the tool will be evaluated through interviews with the instructors. Data on how often the tool's suggestions are overridden and how long it takes the instructor to grade each solution will also be collected. Student experience with the tool will be evaluated through online surveys. Student learning will be evaluated through posttests given each week. The project is expected to improve student learning and instructor productivity in the WPI Algorithms course. It is also has potential to contribute to the broader fields of machine learning, natural language processing, and education through the study, generation, and deployment of effective feedback. The NSF IUSE: EHR Program supports research and development projects to improve the effectiveness of STEM education for all students. Through the Engaged Student Learning track, the program supports the creation, exploration, and implementation of promising practices and tools. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.