US-French Collaboration: Auditory computations for interpreting and producing communication signals.

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Human speech and animal communication require both the extraction of meaning from sound and the processing of one's own voice to guide the production of these vocalizations. These processes require non-trivial computations that have challenged linguists and engineers but that are performed effortlessly by our brains. To understand what are the neural computations performed to decode the behavioral meaning and vocal gestures of communication signals, this study will examine how the auditory cortex of a songbird processes the complete vocal repertoire of its own species. The Theunissen Lab acquired a unique database of all the vocalizations emitted by adult and juvenile, and both male and female zebra finches. This database contains the complete repertoire with multiple exemplars of each vocalization type for many individuals. Because the behavioral context of each communication sound was carefully recorded, these sounds are classified in meaning categories. This database will thus enable the detailed investigation of how the auditory system extract meaning from vocalizations, while controlling for variability of production within vocalization type as well as between individuals. The approach of this project consists in obtaining neural responses to these communication sounds using advanced neurophysiological recording techniques, and then investigating the neural computations by finding the statistics models that best predict these responses. Multi-electrode arrays will be used to record the simultaneous neural activity of large sets of single neurons in the primary and secondary auditory areas. The response of these neurons will then be fitted using statistical models that incorporate increasing levels of abstraction: from elementary sound features, to vocal gestures and semantic labels. The representation in terms of vocal gestures will be obtained from a reduced physical model of the avian vocal organ. This analysis will not only point out the brain regions that are involved in semantic processing but also the nature of the hierarchical computations that lead to these higher-level representations. The research will also investigate the link between perception and production by directly assessing the role of a motor-based representation of sounds in high-level auditory areas. By combining ethological, neurophysiological and computational studies of acoustic communication in a songbird, the project will establish an appropriate animal model system to elucidate how the auditory cortex extracts and categorizes sound features in order to link sound to meaning. Given the similarities in the anatomy and physiology of the auditory system across vertebrates and the common signal processing problems shared in all vocal communications, this study can also contribute significantly to the neurophysiological understanding of neural mechanisms underlying speech perception. This award is being co-funded by NSF's Office of the Director, International Science and Engineering. A companion project is being funded by the French National Research Agency (ANR).