EAGER: Computer Architectures and Algorithms for Adaptive Human Computer Interfaces

University Of California-San Diego, La Jolla CA

Investigators

Abstract

The ease of use of a human computer interface depends critically on the latency of the system and its ability to adapt to the environment. Differences in lighting, visual appearance and user behavior can significantly alter the input data. Furthermore, the reaction time must be less than 100 milliseconds to appear instantaneous to the user. Achieving high accuracy demands a system that adapts to these changing characteristics while processing a significant amount of data in a short amount of time. We propose a computer architecture for adaptive real-time signal processing systems that combines a general purpose processor with custom hardware. The custom hardware performs the low-level, high-throughput signal processing on the raw signals and feeds them to the processor which performs the high level signal processing and decision making. The processor also executes machine learning algorithms that change the parameters of the low-level processing to adapt them to the current statistical properties of the data. This project will develop a human-computer interface based on audio and video sensors that allows a user to interact with the computer through gestures and voice alone. This requires research advances in computer architecture, embedded systems, signal processing, machine learning and human-computer interaction. The major research challenge is in the integration of knowledge from the different areas to create a functional system. This system will serve as a prototype for novel human computer interactions and will be a foundation for future collaboration between the different fields.