EAGER: Preliminary Explorations for the Development of Responsive Prosodic Behaviors for Interactive Systems

University Of Texas At El Paso, El Paso TX

Investigators

Abstract

This EArly Grant for Exploratory Research is motivated by the attractiveness of spoken interaction as a way for people to interact with autonomous intelligent systems. When systems talk to people, speech can convey not only lexical information but also meta information, such as whether the information requires immediate attention or is just background, how well the system understands the user's goals and situation, and whether the system needs to continue providing information or is done for the moment. In human-human dialog such meta-information is mostly conveyed by prosody (intonation): subtle variations in the pitch, energy, rate and timing within utterances. Unfortunately this sort of expressiveness for agents is not today available without using pre-recorded prompts or hand-crafted synthesized utterances, neither of which is flexible enough for systems operating in contexts where the possible configurations of communication needs are not fully predictable ahead of time. Thus there is a need to make spoken language technology better support interaction for robots and other systems. Advances in prosody modeling can also inform language teachers and communication teachers, and health practitioners who deal with communications disorders. This project focuses on two preliminary steps to the goal of using prosody in human-computer dialog: first experimentation with different models of speech synthesis to determine which can best support the realization of prosodically well-adapted utterances, and to determine where improvements need to be made, whether to the engine itself or to the training-data collection procedures; and second a pilot data collection and analysis to determine which aspects of context need to be measured and represented to support appropriate prosodic choices.