HCC: Small: Collaborative Assistants for Team Activities in Virtual Environments

University Of Rochester, Rochester NY

Investigators

Abstract

This project will design and implement prototypes of intelligent collaborative assistants that help human users participate in joint activities as a member of a team. These teams can include both human and non-human (robotic or software) agents collaborating to perform tasks and solve problems. Examples of such tasks include helping the leader of a search-and-rescue team coordinate the search of collapsed buildings following an earthquake or searching for trapped miners following a mine accident, helping coordinate a team responsible for ongoing surveillance and anomaly detection in some area, helping manage a team of agents exploring a distant planet, or helping an analyst locate, monitor and interpret various streams and sources of information in support of a decision or activity. This research will extend prior work in several significant ways: (1) integrating and extending a rich model of teams and activities; (2) increasing the role of knowledge and reasoning in driving the behavior of the collaborative agent in a team setting; (3) extending a model of collaborative problem solving to support human control of agent teams; and (4) developing a feature-rich, interactive, simulated environment based on videogame engine technology for development, demonstration, and evaluation of human-agent teams and assistants. Collaborative assistants have already shown promise in a number of different application domains such as learning and automating tasks on the web or providing natural, effective interfaces for patients to access their personal health information. Several researchers have applied the "conversational assistant" paradigm to domains ranging from customer service agents to embodied virtual agents for education and training. But these assistants have generally been strictly one-on-one, communicating solely with the user and typically responsive only to the user's utterances. They are also generally "face-to-face," in that the participants are located next to each other and a shared communicative state is more or less assumed. This research will greatly broaden the applicability of intelligent collaborative assistants to more complex tasks with multiple participants in more realistic settings. Because the approach is based on a general model of collaborative assistants, the advances from this work will improve the capabilities of collaborative assistants in general. The game-engine-based human-agent team environment developed in this research will be built from freely-redistributable components and will be made freely available to the community for use in education and research. This environment will be suitable for use in undergraduate classes, allowing students to see algorithms and techniques in action controlling agents in a realistic world that can also include humans. Researchers should also find it a useful testbed for experimentation and evaluation. The project will involve both graduate and undergraduate students with a wide range of skills, interests, backgrounds, and personality types, not just hardcore gamers.