CAREER: The Time Course of Bottom-up and Top-down Integration in Language Understanding

University Of Connecticut, Storrs CT

Investigators

Abstract

Context changes the way we interpret sights and sounds. A shade of color halfway between yellow and green looks more yellow when applied to a picture of a banana, but more green when applied to a lime. An acoustic pattern halfway between "p" and "b" is interpreted as "p" following "sto-" but as "b" following "sta-". But does context actually alter perception of sights and sounds, or only their interpretation? Cognitive scientists have long debated when and how "bottom-up" input signals (such as speech) are integrated with "top-down" information (context, or knowledge in memory). Do early perceptual processes protect a "correct," context-independent record of signals, or do perceptual processes immediately mix bottom-up and top-down information? One view is that accurate perception requires early separation of bottom-up and top-down information and late integration. An alternative is that early mixing of bottom-up and top-down information would make systems more efficient, by allowing context to immediately guide processing. In studies of language comprehension, this timing question is unsettled because of conflicting evidence from two measures of moment-to-moment processing. Studies tracking people's eye movements on objects upon verbal instructions support immediate integration: helpful information appears to be used as soon as it is available. Studies using ERPs (event related potentials, which measure cortical activity via scalp electrodes) suggest delayed integration: early brain responses appear to be affected only by bottom-up information. Results from the two measures have been difficult to compare because they have relied on very different experimental designs. In the proposed research the investigator will study the timing of top-down integration in human sentence processing using experimental designs that allow simultaneous comparisons of eyetracking and ERPs, with the goal of determining when and how top-down context is integrated with bottom-up signal information. The proposed work has important implications for the design of language technology. In contrast to computer systems, humans efficiently exploit top-down context, and quickly learn to adapt to new contexts. An obstacle to making computer systems as adaptable as humans is that we do not fully understand how humans balance bottom-up signals and top-down context. The proposed research also has implications for understanding and treating language impairments. For example, understanding how normal perceivers balance and integrate signal and context may help identify subtle bottom-up impairments that lead to unusual reliance on context. The investigator is committed to integrating research and training activities in this CAREER project, and will actively involve undergraduate and graduate students in the research. The investigator will also develop courses designed to prepare students for independent research by providing hands-on training in cognitive theories and time course methodologies.