Neural basis of optimal cue combination: theory and experiments

University Of Rochester, Rochester NY

Investigators

Abstract

The nervous system can sense the world through a variety of sensory modalities providing complementary as well as redundant information about various aspects of the real world. Through a process known as multisensory integration, these sensory signals are merged and used to perform a variety of tasks, such as locating objects in space or identifying the words uttered by a speaker. With NSF funding, Dr. Alexandre Pouget is conducting a strongly theory-driven research program, combining modeling and human studies to investigate the neural basis of this process. The experiments involve asking human subjects to locate their arm in space using multisensory cues. The researchers seek to learn whether subjects can integrate over time three types of cues: visual, auditory and proprioceptive. The optimal strategy in this situation is to compute a weighted sum of those cues, with weights inversely proportional to the reliability of the cues. Intuitively, this makes sense: The system should put greater weights on cues that are more reliable. The funded experiments will determine whether subjects perform this weighted average, and whether those weights adapt when the reliability of the cues is changed from trial to trial. The models will be used to predict human performance and to develop a theory of optimal multisensory integration in neural circuits. The approach relies on a general theory of computation known as Bayesian inferences. This theory has been applied to a variety of problems ranging from sensory processing, high level reasoning (like medical diagnosis), and motor control. Therefore, understanding the neural basis of Bayesian inferences in the context of multisensory integration will have wide implications for general theories of how the brain works, with applications to computer vision and robotics. The Broader Impacts of this project include the development of a course in computational neuroscience at the University of Rochester and opportunities for undergraduate and graduate students to participate in research. This project will also foster collaboration with a laboratory in France directed by Dr. Jean-Rene Duhamel.