EAGER: A Research Infrastructure for Analyzing Speech-based Interfaces

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

The research community's understanding of the speech-to-text problem has reached a point at which most challenges can in principle be met, given a baseline system, enough data from the target domain, and an expert, who knows how to develop or adapt a recognizer for the target context-of-use. Unfortunately, this approach does not scale: despite the growing interest in speech-user interfaces, there are a limited number of experts equipped to analyze and develop an accurate speech recognizer. This Early Grant for Exploratory Research explores the possibility of formalizing a speech recognition expert's implicit knowledge of the required analysis and development steps in a rule-based knowledge base, which can help a speech recognition non-expert develop a speech recognizer as part of an application, such as a dialog system in a rare dialect. Speech recognition experts adapt and improve recognizers by listening to data, aggregating error reports, and then adjusting parameters, retraining models, or applying adaptation techniques, based on their assessment of the mismatched context of use. This project extracts intuition from contextual interviews with such experts, develops a proof-of-concept expert system to predict the gains a system would see from specific adaptation techniques, and explores the factors which will make this approach feasible. This project creates ways to make development of speech-enabled applications more accessible to a broader class of researchers, students, and practitioners, particularly from the user interface area. It will make joint development of user interface and speech recognition feasible, without requiring large teams with varied skill-sets.