Coordinating Information in Sentence Processing

University Of Rochester, Rochester NY

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): This research monitors eye movements to objects in a visual workspace, which are closely time-locked to language production and comprehension, as participants perform natural tasks, such as following producing and following instructions, and conversing to solve problems using the objects. The goal is to understand how listeners coordinate linguistic and non-linguistic information as they understand language. Coordinating information is central to language comprehension because speech unfolds as a rapidly changing transient acoustic event. As a consequence, listeners encounter ambiguities at multiple levels of linguistic representation (e.g., phonetic, lexical, syntactic, semantic/discourse model, etc.). Many ambiguities are temporary in that subsequent linguistic information soon resolves the ambiguity; others are resolved by appeal to non-linguistic context-the particular circumstances under which an utterance is heard or read; others by appeal to real-world knowledge. The ambiguity problem is pervasive: even unambiguous words and phrases are temporarily ambiguous. Yet, comprehenders quickly coordinate multiple sources of information to arrive at provisional interpretations as the speech unfolds. The results of this research are used to develop and evaluate explicit mechanistic models that support real-time comprehension and can account for the major empirical phenomena. These models are essential for understanding normal comprehension and its development, developmental disabilities, and comprehension impairments that may arise from brain injury. The results and methods that arise from this investigation are also becoming increasingly relevant to informing and evaluating computer-based spoken dialog systems. These systems are beginning to find applications in health-related domains (e.g., systems that interact with humans using spoken language to answer queries) and prototypes are under development that offer advice, for example, in coordinating rapid response to medical emergencies.