IRES: Avatar-based Adaptive Context System

The University Of Central Florida Board Of Trustees, Orlando FL

Investigators

Abstract

Part 1 This interdisciplinary partnership between the University of Central Florida and the Fraunhofer Institute for Digital Media Technology (FIDMT) in Ilmenau, Germany is focused on adaptive conversational avatars, the rapidly emerging field crossing computer engineering, computer science, education, communications, and social science. Immediate applications of this research field include artificial intellegence and national security (including cyber-security), interactive robotics, improvement of quality of life for disbaled, and health and caretaking for children and elderly. This project will place students from the University of Central Florida under the mentorship of the PI (Dr. A. J. Gonzalez) and of Dr. Klaus Jantke, the counterpart at FIDMT in Ilmenau, Germany. Dr. Jantke is the director of the Children's Media Department of FIDMT located in Erfurt, Germany and has a long and illustrious history in research in computing media. The international aspect of innovative and advanced research is essential in modern research hence the PIs will work with three cohorts of students, one during each year of the project's existence. Each cohort will include one graduate student and either two undergraduates (the first year) or four (in each of the subsequent years). The research period for each cohort will be 16 weeks - eight weeks in the US and eight weeks in Germany for each year of the grant period. This research project is motivated by an ancient art of storytelling. In our pursuit of an artificially intelligent computer agent, the IRES project seeks to build a capability to autonomously synthesize possible scenarios for the system development and to modify them dynamically upon listener request. More specifically, the topic of the research in this project is the creation of an avatar-based system that can synthesize and adapt a scenario according to the user's request in real time and without any pre-scripted pathways. Good storytellers were treasured in medieval times, given the lack of other media through which to relate a story to a mostly illiterate population. Therefore, the project seeks to embody the storyteller in a lifelike avatar that resembles an actual person. This avatar will tell the story to the listener in spoken natural language, and interact with her/him when the latter requests changes to the story. Part 2 Storytelling media have evolved over time, from oral stories to modern E-books. Since the development of the computer, storytelling systems have become a science of their own, and have evolved from simple systems that can only generate a single short story to systems that respond to the listener's actions by modifying the story dynamically in real time. Digital storytelling has therefore become a growing field within artificial intelligence. The project seeks to take this evolution of storytelling media one step further by doing research to create a virtual storyteller who tells a dynamic story. The story is modifiable through a request by the listener (typically a child, a student, or an elderly person), yet will seek to remain realistic as well as interesting. Every story has a story space. That is, only so many things can happen in a story. We use contextual reasoning to represent the story space. In the real world, courses of action are influenced by the current context, making some conversational avatars very attractive while others unattractive when addressing the current situation within the story space. In a similar manner, the situation faced by the protagonist in the dynamic scenario will limit the choices of actions that he/she would otherwise have, thereby taking the story in various directions, none of which need be specifically pre-scripted. The PIs base the proposed research on the use of formal methods to manipulate the story space within the main theme of the story. By formal methods the PIs mean that one represents the story knowledge formally in terms of strings, interaction sequences such as storyboards and graphs, formulas (for conditions), and the like. Formal methods, therefore, will give the ability to reason with formal methods (string comparison, unification, anti-unification and the like) in the story space. Formal methods have been used in the literature to manipulate contextual information.