Building A Teacher-AI Collaborative System for Personalized Instruction and Assessment of Comprehension Skills

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Students need fundamental literacy skills, especially reading comprehension, to successfully engage in STEM learning and careers. A number of studies have explored the use of AI technologies, such as chatbots, to improve students' reading comprehension by engaging students in interactive dialogue during reading. This approach is particularly promising for younger students who are in a critical period for developing reading skills. However, scaling up these AI resources to make them accessible and relevant to diverse learners and instructors remains a challenge. This project aims to harness recent advancements in AI, particularly large language models, to enable teachers to collaborate with AI in creating interactive reading resources that ask students questions, listen to and interpret student responses, and provide tailored feedback to students during reading, focusing on students from kindergarten to second grade. This will allow teachers to contribute their expertise to develop AI resources tailored to their students' needs. The project will shed light on whether the extensive manual labor involved in chatbot development, typically performed by content creators, designers, and engineers, can be significantly reduced through AI-teacher collaboration, and whether the resulting chatbots effectively support teachers' instruction and promote students’ reading comprehension. This project will be carried out in four stages. This first stage involves the development of innovative AI models to automatically generate question-answer pairs based on reading materials teachers select. The models will be tailored to meet the unique requirements of educational contexts. In the second stage, a user-friendly teacher-AI collaborative system will be developed through a contextual inquiry and participatory design process. This system will enable teachers to verify and modify the question-answer pairs generated by AI and subsequently incorporate them into a chatbot that engages students in dialogue. Teachers' modifications to the question-answer pairs will feed back to the system so that the AI models can gradually learn and adapt to each individual teacher's preferences. In the third stage, the research team will develop the chatbot's capability for adaptive interaction so that it can carry out dialogue and provide scaffolding based on both the accuracy and sentiment of students' responses. The fourth stage will involve an examination of the usability and effectiveness of the teacher-AI collaborative system and resulting chatbot in supporting personalized instruction and assessment. To this end, the research team will carry out a field test involving an under-power randomized controlled trial. Five teachers and their approximately 150 students will be recruited to participate, with half of the students in each class randomly assigned to read interactive texts with a chatbot generated by their teacher while the other half reads the original text without the chatbot. Observations and interviews with teachers and students will shed light on the usability of the teacher-AI co-created interactive reading materials. Students' post-reading comprehension will be assessed to provide evidence on the system's educational impact. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.