CRII: CHS: Designing Scalable Help Tools for Programming Courses

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

The growing demand for programming skills has led to an influx of new learners in both traditional and online programming courses. However, it is challenging to provide effective personalized help, particularly online. Synchronous tools such as video chat enable interactive help but limit scheduling flexibility, which is a key advantage of online courses. Asynchronous tools such as discussion forums provide flexibility and allow students to learn from prior questions but are less interactive and lose important contextual information about the learner's question. This project aims to design, build, and test "semi-synchronous" tools to combine the best of both help styles. The idea is that capturing snapshots of the program's code and output should allow learners to ask clearer questions and let instructors form a more complete picture of the problem that should improve their answers, while integrating chat features with the code and tracking how code changes as the discussion progresses should help other learners make more effective use of prior questions. Building and testing these tools will advance knowledge about how to develop scalable collaborative systems, useful information repositories, and more effective support resources for programming courses. Further, it will have practical impacts on software education, particularly for students with lower socioeconomic status who could benefit most from better support in online courses. The project is organized around the evolution and evaluation of an existing prototype semi-synchronous tool called chat.codes. On the development side, there are two major activities. The first is integrating version tracking with the existing chat interface and developing tools to visualize how code changes along with conversational turns during a help session. The second involves developing tools to archive, search, and view past conversations based on specific discussion topics, code keywords, or instructor-created tags. The evaluation has four main phases. The first is a one-hour lab study pairing students and instructors on simulated programming tasks that require students to learn a new concept from the instructor, aimed at evaluating the usability of chat.codes relative to other synchronous help systems. The second is a lab study to test the archiving, search, and visualization features, assigning students to a programming problem and giving them access to the help sessions from the first experiment but not a live instructor to see how well the archived conversations both help solve the programming problems and convey the underlying concepts. Results from both studies will be used to improve chat.codes, which will then be deployed as a remote support tool for two semesters in a large in-person introductory undergraduate programming course. This deployment will be open-ended and students and instructors will not be required to use the tool but will be encouraged to do so; the project team will use a combination of interviews where they ask users to compare chat.codes to other support tools, along with analysis of logged help sessions, to better understand the strengths and weaknesses that come with integrating the code and chat elements. The team will then develop a final version of the tool that will be deployed alongside existing online programming courses. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.