Understanding and Mitigating the Impacts of Code Intelligence Systems in Introductory Programming Courses

University Of Houston, Houston TX

Investigators

Abstract

This project aims to serve the national interest by understanding the impact on computer science education of using artificial intelligence (AI) to generate computer code. Artificial intelligence tools can generate computer code from the description of a problem written in natural language, and it may lead to faster and easier development of high-quality software programs. While these “code intelligence” (CI) systems may positively improve workflow for professional developers in industry, they also have potential to impact how students learn computing. This research will investigate the impacts of the use of CI systems to analyze students’ tinkering behaviors, which are associated with learning gains, and students’ feelings of self-efficacy, which affect performance outcomes. These systems may help struggling students learn, or they may shortcut student learning if the students become too reliant on them. CI systems can pose serious challenges for instructors in designing novel course assignments or identifying plagiarism in the learning environment. In this research supported by the BCSER program, the PI will develop proficiency in learning theories and research methods to study the impacts of the adoption of CI systems on learning in introductory college programming courses. The outcomes of this research will benefit student retention and diversity in computer science education. This project will utilize extended concepts from learning science knowledge to evaluate and characterize the impacts of code generation tools on students' learning in introductory programming courses. The first project goal is to develop the necessary infrastructure to use intelligent code generation within a web development environment for students, so that students have access to code-generating capabilities. The second project goal is to understand the use of the code generation tool in introductory computer science courses through engagement in pilot experiments examining effects on tinkering and self-efficacy. A third project goal is to design and evaluate an intervention to mitigate the potential negative impacts on students’ learning by encouraging students to pay closer attention to the generated code. These will create an extended knowledge base for positive use of CI systems. A fourth key goal of this project involves engagement in professional development workshops and online classes related to computer science education research as well as holding formative assessment meetings with mentors. These will improve capacity building and sustainability of computer science education research for the PI and the institution. The computer science research community will gain access to lessons learned, challenges faced and products developed, including corresponding datasets and reproducibility packages, through publications and presentations at computer science conferences. The project is supported by NSF's EHR Core Research Building Capacity in STEM Education Research (ECR: BCSER) program, which is designed to build investigators’ capacity to carry out high-quality STEM education research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.