EAGER: Building a Foundation for Hands-on STEM Learning at a Distance: Pedagogical Agents for Embodied Education in Virtual Reality

University Of California-Davis, Davis CA

Investigators

Abstract

Maker education holds significant promise to revitalize and diversify education in mathematics, science and engineering, with its combination of low barriers to entry for sophisticated technical practices, a rich community infrastructure of resources and support, and a playful learning mindset. However, maker education is best done in person, where the learner and facilitator can discuss the artifacts being built, each can manipulate those objects, they can use gesture to explain concepts and they can freely adjust their viewpoint on the workspace. This makes it challenging to scale maker education, because it is difficult to provide effective remote instruction and difficult to create instructional resources. This proposal will develop embodied pedagogical agents -- animated characters in a virtual environment -- that can provide facilitation on maker tasks, focusing on electronic circuit design. Research will begin by building a corpus of interactions in embodied virtual reality, where expert facilitators and learners appear as animated avatars with live tracking to drive their movements and interact to build electronic circuits. Phase two will build a prototype pedagogical agent based on the corpus analysis and data. In the near term, the project will provide direct educational benefit to youth from groups underrepresented in STEM and will strengthen university-community partnerships. Over longer time scales, the development of an empirically grounded catalog of facilitation moves has potential to inform teacher education and pedagogical agents can better support remote learning. The approach is scalable to a range of tasks that rely on embodied interaction, such as skills training or physical therapy. The proposal will make significant contributions to both learning and computer science. As maker facilitation often relies on intuitive decisions by the teacher, it is not a well understood process. The corpus will provide important data on the verbal and nonverbal strategies employed to facilitate Maker education. Analysis of the corpus and later agent-learner interactions will improve our understanding of facilitation, the role of nonverbal communication, and how learning is guided in these settings. The technical work to build the agent will establish the utility of pedagogical agents in this domain. The corpus will also provide a unique and rich dataset on which to develop future machine learning algorithms for behavior synthesis. The dataset will include verbal and nonverbal behavior, along with a full coding of the environment and interaction objects and teaching moves applied by the facilitator, exceeding anything currently available. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.