Collaborative Research: Towards Life-like Computer Interfaces that Learn

The University Of Central Florida Board Of Trustees, Orlando FL

Investigators

Abstract

This collaborative project, developing and evaluating lifelike, natural computer interfaces as portals to intelligent programs in the context of Decision Support System (DSS), aims at providing a natural interface that supports realistic spoken dialog, non-verbal cues, and the capability of learning to maintain its knowledge current and correct. The research objectives focus around the development of an avatar-based interface with which the DDS user can interact. Communication with the avatar takes place in spoken natural language combined with gesture expressions or by pointing on the screen. The system supports speaker-independent continuous speech input as a spontaneous dialog within the specified DSS domain. A robust backend that can respond intelligently to the questions asked by the DDS user is expected to generate the responses spoken in reply by the avatar with realistic inflection and visual expressions. The work develops, prototypes, and evaluates the desired user interface capabilities by using the model of a program officer to create a realistic avatar that can answer users' questions and respond in a humanly natural manner. The project extends a current sponsored project on information gathered related to a centers program where a program officer serves as subject matter expert. The recently-developed AlexDSS system that answers questions to users about the I/UCRC program provides the baseline intelligent system behind the avatar. The avatar interfaces are targeted for both general users as well as for experts responsible for updating/correcting the domain knowledge therein. The work represents a collaborative project between the Intelligent Systems Laboratory (ISL) at UCF and the Electronic Visualization Laboratory (EVL) at UIC. The EVL team focuses on avatar development encompassing Visualization and Interaction with Realistic Avatars and Evaluation of System Naturalness and Usability. The ISL team concentrates on Natural Language Recognition and on Automated Knowledge Update and Refinement.