HCC: Small: Designing Effective Gaze Mechanisms for Cross-Modal Embodied Agents

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

The goal of this project is to design and build gaze mechanisms for embodied agents that can achieve high-level social and communicative goals that people achieve using such mechanisms and that can be applied across a wide range of agent presentations and task domains. Embodied agents promise significant social, cognitive, and organizational benefits through applications in education, training, rehabilitation, and collaborative work. However, in order to be effective across a wide range of applications, agents must be able to use the nonverbal social cues that humans employ in their communication and to employ these cues in whatever modality the agent is presented in - whether it be a social robot, a life-sized virtual human, or an animated avatar on a portable display. Gaze cues are particularly important social signals. Although they are subtle, they can serve as powerful mechanisms for achieving high-level social and communicative goals, such as improving a listener?s comprehension, controlling the flow of a conversation, and indicating interest in or appraisal of objects. This project investigates how such mechanisms might be designed and built for embodied agents and how similar social and communicative goals could be achieved using different agent representations across different task domains. This investigation will involve (1) performing formal observational studies to better understand how people use gaze, (2) developing computational models that synthesize gaze behaviors that can be controlled precisely and retargeted to a range of agent platforms, and (3) evaluating in experimental studies the effectiveness of using gaze cues across a range of agent presentations and task contexts. Success in this project will create new knowledge on human gaze behaviors, connecting the high-level findings in the social science literature to more detailed, low-level cues and mechanisms. It will also produce a set of techniques that are based on this understanding for synthesizing controllable and flexible gaze movements that agents can use in order to achieve social and communicative goals. Finally, it will validate the effectiveness of the use of gaze cues by agents across a variety of agent presentations and task contexts. A trans-disciplinary approach will combine rigorous, formal observational studies to build detailed models of human communicative mechanisms with practical efforts to build usable computational models that meet the needs of creating agents that work in real-world tasks. The focus on human models insures that the computational models are well founded, while the focus on developing practical algorithms guides the human studies towards creating understanding that will be most informative for agent design. The project plan involves connecting the disparate communities that work on developing social agents and training students to do the trans-disciplinary work required to create effective embodied agents. The project will also enable K-12 outreach efforts to use robots and connections to social science to engage students and increase participation by under-represented groups.