Neural Basis of Active Perception in Natural Environment

University Of Southern California, Los Angeles CA

Investigators

Abstract

Understanding how animals perceive and act upon complex natural environments is one of the most pressing challenges in neuroscience, with applications that have potential to revolutionize not only our understanding of the brain, but also machine vision, artificial intelligence, and robotics. Until now, studying the neural basis of active vision - how visual stimuli give rise to eye movements under diverse task conditions - has largely been restricted to simplified laboratory stimuli, presented to overtrained animals performing stereotypical tasks. With funding from the National Science Foundation, the Canadian Institute of Health Research, and the National Geospatial Intelligence Agency, Dr. Douglas Munoz at Queens University in Canada and Dr. Laurent Itti at the University of Southern California will combine neurophysiology and computational modeling to investigate free viewing in natural environments. Using multi-electrode arrays, this project will record in a deep brain structure, called the superior colliculus (SC). The SC is a layered structure comprising several well-understood neural maps, from purely sensory representations in the superficial layers, to sensorimotor representations linked to the control of eye movements in the deeper layers. The project will start by characterizing responses of neurons in the SC under simple stimulus conditions: When the animal is simply looking at a central fixation cross on a display while small isolated patterns are presented at other visual locations; when the animal searches for an oddball item among an array of distracting items; and when the animal inspects natural images and video clips. The project will extend the investigators' salience map theories and models, and develop a new model of the SC. The complete model will predict, from any image or video clip, which visual locations are more salient, task-relevant, and candidate targets for eye movements. The project leverages a cross-disciplinary collaboration between a neurophysiology lab (co-PI Douglas P. Munoz) and a computational modeling lab (PI Laurent Itti). This will allow, through the combination of experiments and modeling, the interpretation of an otherwise undecipherable mass of data collected during natural viewing. Conversely, the theories will guide further experiments. Coupling multi-unit recording with modeling during free-viewing of natural videos has never been attempted before, and it is expected that it will lead to new understanding of how percepts map into actions under natural conditions. The project will support undergraduate and graduate students, and post-doctoral researchers, who will benefit from exposure to combined physiological and computational techniques, as will the investigators' teaching. In addition to publications, all theory and algorithm source code will be freely distributed, and data will be available through the CRCNS data sharing web site. This research is hence expected to lead to new and broadly accessible fundamental advances in the understanding of how animals use visual information to guide behavior, and how one could build machines which act in similar ways when faced with the complex natural world.