Designing Next-generation Mobile Interfaces for Dynamic Conversational Speech

Oregon Health & Science University, Portland OR

Investigators

Abstract

Interactive Multimodal Interfaces: Designing for Human Performance This is a creativity extension to the PI's continuing award. The PI plans to conduct a study involving mobile testing of her multimodal system. Subjects will be a mixture of adults and 7-to-9-year-old children. The goals include: (1) establishing the full research infrastructure needed to support extensive mobile testing and semi-automated data analysis; (2) documenting mutual disambiguation during mobile testing of a multimodal system, and studying the factors associated with its enhancement; (3) examining the relation between system recognition performance in a mobile environment and users' signal characteristics, ambient noise levels, and signal-to-noise ratio information; and (4) exploring mobile speech signal patterns and system recognition performance in diverse user groups. The study will provide critical information about multimodal interface designs appropriate for supporting robust mobile use in real-world contexts and by varied users, and in particular for creating adaptive multimodal architectures that are capable of monitoring the environment on a command-by-command basis and adapting mode weights intelligently to avoid recognition failure and stabilize system performance. The data collected during the use of the PI's mobile system will assist in identifying a variety of new research issues and interface design challenges that have neither been recognized nor probed by the broader research community