EAGER: Collaborative Research: Modeling Distinctive Partners in Adaptive Spoken Dialog

Suny At Stony Brook, Stony Brook NY

Investigators

Abstract

The human ability to use language flexibly is a hallmark of robust intelligence. In interactive dialog, utterances are dynamically tailored to the common ground or specific context with specific partners. However, interaction with spoken dialog systems is highly constrained and constraining, allowing speakers very little flexibility in what they can say while the system presents pre-determined messages. To make interactive dialog technology broadly useful, this exploratory interdisciplinary project collects a corpus of dialogs exhibiting some important sources of variation, analyzes the corpus, and uses the resulting analyses to develop models and prototype implementations of dynamic dialog strategies. The ultimate goal of this effort is to support the synthesis of entirely new, flexible, and robust spoken dialog systems that are capable of adapting on-line. The Walking-Around corpus consists of 40 human-human dialog interactions where a remotely located person gives directions to a pedestrian walking around in an urban or campus environment. The experimental paradigm varies the friendship relationship of the dialog partners, whether the director can see what the pedestrian sees, and the familiarity of both the director and the pedestrian with the environment. No other existing direction-giving corpora model dialog interaction in an outdoor real-time environment where the physical context grounds the dialog context. The resulting corpus is used to test hypotheses about, and develop models of, the evolution of local and global dialog adaptation strategies. Key to our effort is determining which adaptations are actually functional, that is, beneficial for a particular task or context in spoken dialog systems.