Automated Feedback in Undergraduate Computing Theory Courses

Rochester Institute Of Tech, Rochester NY

Investigators

Abstract

Computing theory poses and answers questions such as "Which problems are efficiently computable and which are not?" Answering such questions is important for any computer scientist and for any kind of software development. For example, it is better to determine if a problem is computable before spending a lot of time trying to write a program to solve it. Unfortunately, many students struggle with computing theory, because it is more abstract and mathematical than other computer science topics. As in any other knowledge area, students need to practice to get better at computing theory. A problem is that feedback on their work is not immediate and, while students wait for feedback, they stop interacting with the material. They may have to wait for days, since grading an assignment often takes a lot of instructor time and the instructors may have many assignments to grade. This project will increase the speed and, potentially, the quality of feedback to computing theory students by developing an automated feedback tool. The feedback will tell students whether a solution is correct or not, a convincing reason why an incorrect solution is incorrect, and provide information about the quality of a solution. Students will be able to use this immediate feedback to improve their solutions, get more practice, and increase their understanding of the material. In addition to building the feedback tool, this project aims to conduct research on the feedback tool's effectiveness. This project has the potential to contribute to the education of a strong computing workforce and to support development of students' independent learning skills. Although understanding computing theory concepts is very important, it is challenging. Typically, as a first step, students in computing theory classes learn about various models of computation. To understand more complex computational issues, students need to fully comprehend the possibilities and limitations of these models. JFLAP (Java Formal Languages and Automata Package) is a widespread tool that provides a way for students to interact with these concepts. However, like other interactive tools in this area, it does not provide detailed feedback on student solutions. This project will build a feedback and grading tool on top of JFLAP, to increase the likelihood that the feedback tool will have broad applicability. To accomplish this goal, the project will develop and evaluate the tool in the context of three research areas: (1) Computer Science Education: Do students who use the tool understand theoretical computer science concepts better than students who do not use the tool? (2) Theoretical Computer Science: How can software generate a convincing reason for why a student solution is incorrect? and (3) Artificial Intelligence: How can feedback be given about the quality of a student's solution? The project's research and software development activities will involve ten undergraduate students, who will be recruited with emphasis on including women and deaf/hard-of-hearing students. Thus, the project will directly contribute to these students' scientific and professional development. Project outcomes will be disseminated at scientific conferences and workshops, as well as at the University's innovation fair, which is attended by 35,000 visitors, including middle and high school students. Developing the feedback tool and completing research on its effectiveness has the potential to improve instruction and learning of computing theory. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.