CRCNS: Neural Reprensentation of Hierarchical Visual Concepts in Natural Scenes

Carnegie-Mellon University, Pittsburgh PA

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): The complexity of natural images is potentially enormous: the number of possible images that can be described by a smallish (100 by 100 pixels) picture is practically infinite (10000256), more than all the images the human race has ever witnessed during its entire existence. How can any system process input data of this magnitude of dimensions and interpret/understand it in terms of the estimated 200,000 objects in the world, their spatial layouts, and scene structures? Yet, this is a task that human visual systems routinely perform in a fraction of a second. The secret must lie in the fact that natural images are highly redundant, living in a restricted space inside this universe of almost infinite possibilities, and that mammalian visual systems have discovered and exploited this fact. In particular, we conjecture that neurons and populations are tuned to the statistical structure of natural images, building on previous work showing, for example, that sparse coding ideas can help predict receptive field properties of 'simple cells' in the visual cortex. This proposal has three stages. Firstly, we will perform a statistical analysis of natural images to classify and model the types of visual patches that appear. This will result in a stimulus dictionary, which will be used as stimuli to investigate the tuning properties of neurons and neuronal populations, and a visual concept dictionary which will be used to make predictions for the tuning properties. Secondly, we will perform multielectrode neurophysiological investigation of the tuning properties of neurons, and neuron populations, at different levels of the visual cortex in response to the stimulus dictionary. Thirdly, we will perform data analysis to model the tuning properties of neurons and populations using a combination of model-driven, which assumes that neurons are tuned to statistical properties of images, and data-driven approaches which can be thought of as learning 'neural visual concepts' directly from the neuron's response to the stimuli. Our theoretical approach - for learning the stimuli dictionary, the visual concepts, and performing data analysis - is based on statistical and machine learning techniques. These assume a hierarchical compositional structure for the data which offers the possibility of taming the complexity of natural images and is also consistent with the known hierarchical structure of the visual cortex. Intellectual merit: This research will help understand the structure of natural images, determine models for the tuning properties of neurons in the visual cortex, and develop novel data analysis techniques. It has the potential to significantly advance our understanding of the statistical structures of natural images and the neural encoding of these structures, including the population level. This will lead to greater understanding of the visual cortex and also help the development of computer vision systems. Broader impacts: This project is interdisciplinary in nature and should have broad impact in multiple disciplines: neuroscience and biological vision, statistical neural data analysis, computer vision, and machine learning. Understanding neuronal properties in the visual cortex is a pre-requisite to the clinical enterprise of developing therapeutic methods and prosthetic devices for the visually impaired. The proposed research program will help facilitate a new graduate program in Computational and Cognitive Neuroscience at UCLA, an inter-college undergraduate minor in Neural Computation at CMU that the investigators are developing at their respective universities. The investigators also plan to organize workshops in NIPS, COSYNE, as well as to integrate their research into both undergraduate and graduate curriculum in their respective universities. This work will also affect undergraduate students at other colleges, by a summer undergraduate training program in Pittsburgh, another at CMU's Qatar campus. In addition, we will propose a workshop and summer school at IPAM (UCLA). We anticipate that this research will lead to invited lectures, peer reviewed publications and, if successful, will have national and international impact. The PIs have good track record in involving undergraduates, including women and minorities, in their NSF-sponsored research, and will continue to endeavor in the training of the next generation of computational neuroscientists.