The interaction of auditory and visual detection, discrimination, and recognition in infants’ real-world listening environments

University Of Washington, Seattle WA

Investigators

Linked publications & trials

Abstract

ABSTRACT Infants are more susceptible to noise than adults, and learn speech and language in noisy environments. Visual speech could be one of the most important cues available to infants to compensate for noisy listening environments, but a coherent picture of audiovisual (AV) speech perception development has yet to emerge. Multiple mechanisms underlie AV benefit to speech perception in adults: common AV onsets and offsets provide information about when to listen, correlations between the amplitude envelopes of auditory and visual signals help to distinguish between different consonants, and experienced listeners can use visual/multimodal phonetic representations. Differences in the mechanisms underlying AV speech perception benefit across tasks and age groups have lead to inconsistencies in the developmental literature. In order to form a unified account of AV speech perception development, this research focuses on the development of the mechanisms underlying AV speech benefits. The proposed research will use a hierarchical framework to investigate AV speech perception. Specifically, the proposed research will compare performance across a hierarchy of perceptual tasks and use stimulus manipulations to limit the AV benefit mechanisms available to participants. The aims are to determine whether typically developing, 7-month-old infants use visual speech cues to improve detection, discrimination, and identification of auditory speech in noise and to investigate the mechanisms by which infants and adults benefit from visual speech. To that end adults and infants will complete auditory-only, visual-only, and AV speech detection, discrimination, and identification tasks, matched on procedural details, but varying in the level of perceptual processing required to perform them. An additional condition uses a visual speech signal that only provides information about the onset/offset of the auditory stimulus. The research uses an established, developmentally appropriate, and psychometrically rigorous behavioral testing method, the observer-based psychoacoustics procedure (Werner, 1995). We expect that (1) infants will detect, discriminate, and identify auditory speech in noise better when they can also see the talker compared to when they can only hear the talker, (2) infants' benefit will primarily result from the visual signal providing information about when to listen, and (3) infants will benefit less than adults on higher-level tasks because adults will use the crossmodal correlations and visual phonetic information to gain additional benefits on higher-level tasks. The proposed research will improve our understanding of how speech sound detection, discrimination, and identification interact with vision in infants and adults, and lead to a better understanding of the development of auditory perception in real-world listening environments. It will provide researchers with a context in which to study development in older children with normal hearing and infants and children with hearing loss. It will also provide the evidence-base for recommendations that parents promote and make use of AV skills in support of a hearing-impaired infant's or child's language acquisition.