Temporal dynamics of phonetic perceptual organization

Barnard College, New York NY

Investigators

Abstract

Scientists and engineers have studied speech to understand why this form of communication is so effective. They have also sought to create speaking and listening devices that approach the accuracy and ease of everyday communication, with modest success. The research problem has been easy to define: English is composed of more than 100,000 words created from over 16,000 different syllables and syllables are composed from a small inventory of several dozen consonants and vowels. Automatic speech recognition would be remarkably easy if these linguistic properties - words, syllables, consonants, vowels -produced uniformity in the sounds that talkers actually make. In fact, each utterance is also physically unique, whether in its sound pattern or in the visible movements of the speaker's face. Different vocal anatomy in men, women and children causes complex variation in sound production even when the linguistic message is the same. Moreover, aspects of a talker's productions may express the dialect and speaking style of their family and linguistic community. Human listeners readily attend to the acoustic hints of these individual and social markers while also listening for the message. This project will examine how these different sources of perceptual information for speech are organized, how they are integrated over time, and how they allow perceptual tuning to the speech of individual talkers. Ultimately, a more complete account of the perception of speech can lead to improvement in recognition technology and to the creation of assistive devices. Three experimental projects will be performed: 1) to estimate the temporal dynamics of auditory sensory integration; 2) to determine the dimensions of exposure-based perceptual tuning to the characteristics of individual talkers; and, 3) to describe and model the intrinsic differences in auditory and visual temporal sensitivity and persistence that affect audiovisual speech perception. In each instance, the perceptual sensitivity to linguistic properties, talker characteristics, and language general features of spoken language will be assayed using discriminating and robust measures of auditory and audiovisual resolution. The studies explore the versatility of perceptual faculties applied to speech and provide an opportunity to identify the principles underlying the remarkably robust perceptual abilities that support and sustain communication. The overall goal is a formal and functional characterization of the cognitive resources that insure the perceptual stability of spoken communication in natural environments, whether the source of speech is visible or not, whether the talker is familiar or not, and whether the quality of the sensory samples of speech is natural or not. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.