Using fMRI to Understand Verbal and Nonverbal Human Communication

Baylor College Of Medicine, Houston TX

Investigators

Linked publications & trials

Abstract

? DESCRIPTION (provided by applicant): Deficits in communication are common to many disorders ranging from autism spectrum disorder to stroke. Speech perception, a critical component of communication, is a fundamentally multisensory process: when conversing with someone, we use both visual information from the speaker's face and auditory information from the speaker's voice. Facial movements are critical for understanding speech because they provide information that is not present in the vocal (spoken) speech we hear. This information may be contextual: eye movements tell us where people are directing their attention, and mouth movements tell us about emotion (e.g., a smile indicating happiness). Facial movements can also be integrated with the auditory information to assist with speech perception in noisy environments: if we are unsure of a word that we heard, we can use the shape of the mouth to help us identify the word. Although speech perception is one of the most important cognitive functions performed by the human brain, we remain largely ignorant of how visual speech cues are processed. The goal of this proposal is to determine how the brain processes facial movements and heard speech, the two key elements of speech perception. The superior temporal sulcus (STS) has been identified as the critical hub for the integration of socially relevant information as it responds strongly to both facial movements and to voices. However, it is unclear how the STS is organized. Understanding the functional organization of the STS will allow us to delineate the mechanisms by which we use facial movements and auditory speech to understand and interact with others. Our overarching hypothesis is that the STS has functional subdivisions that are specialized for processing and integrating these critical classes of stimuli. To test this hypothesis, we will use blood oxygenation level dependent (BOLD) functional magnetic resonance imaging (fMRI). We will use traditional activation-based fMRI, where we look for regional-level differences in average activation, as well as newer information-based techniques such as multi-voxel pattern analysis (MVPA) and encoding models, which allow us to understand the representational content in the observed fMRI activity patterns. If successful, the results of these studies will have implications for our understanding of multisensory integration as well as the potential to guide therapeutic interventions: by pairing speech therapy with non-invasive stimulation applied to these circuits or neurofeedback measurements of brain activity, we could help increase the use of visual speech information in patients with hearing loss or motor speech disorders (e.g., apraxia of speech).