Learning in audio-visual integration of speech

Carnegie-Mellon University, Pittsburgh PA

Investigators

Linked publications & trials

Abstract

[unreadable] DESCRIPTION (provided by applicant): Listeners' perception of speech is dramatically affected by visual information from speakers' faces. Visual information greatly improves speech comprehension in noise and can even alter the perception of clearly audible speech. Visual speech information is particularly beneficial to individuals with hearing impairments and visual speech has recently been used to treat communication disorders; however, the cognitive and perceptual mechanisms that integrate information across sensory modalities are not well understood. A greater understanding of these mechanisms will enable further progress in treating communication disorders. The proposed research will provide new methods for testing theories of audiovisual speech perception by training participants on novel visual speech cues that can be controlled and manipulated in ways that normal visual speech cues (i.e., speakers' faces) cannot. Experimental results and computational models will be used to examine perceptual constraints on audiovisual integration and the development of audiovisual integration over the course of learning. [unreadable] [unreadable]