Visual pattern representation in the extrastriate cortex

New York University, New York NY

Investigators

Linked publications & trials

Abstract

Project Summary / Abstract The goal of this research is to discover the cortical computations that determine the response properties of neurons in visual cortical areas V2 and V4, two of the largest visual areas outside the primary visual cortex (V1) in primates. These areas are critical conduits from the primary visual cortex to temporal lobe areas responsible for the recognition and recollection of visually presented objects and scenes. V2 receives a strong direct input from V1, and depends on V1 for its visual responsiveness. We have developed a two-stage model, in which responses are constructed from a suitable combination of V1 afferents, with the design of each stage following a common canonical form. We have characterized the strengths and limitations of this model, and compared it in detail with a related neural net model trained on natural image data. We will now extend and refine our model by giving it additional capabilities, and we will improve the power of our fitting procedures by using data recorded simultaneously from populations of neurons. The next step is to extend understanding to V4, which receives the bulk of its direct input from V2. We are not yet ready to build a principled model of how V4 combines inputs from V2. We believe that V4’s functional circuitry will be similar to that of V2, but in advance of the complete model for V2 we lack a precise account of V4’s inputs. Measurements of V4 responses demonstrate enhanced representation of complex image features, which encourages us to build a model based on both single-neuron and population responses to account for the way it transforms input from V2. To complement the development of the neural model, we will also develop and extend our neural network model that is trained on natural images. A key component of V4’s response is its selectivity for complex forms. Our measurements of selectivity along the image continuum between natural forms and statistically matched textures demonstrated that information about this continuum is captured by relatively late components of the visual response. We will therefore use reverse correlation methods to explore the nature and dynamics of V4 responses to textures, natural images, and suitable image elements. We will also study responses to distorted natural images generated by a diffusion model trained on natural images, which embody an implicit representation of the high dimensional space that those images occupy. This will allow us to explore whether and how neurons in V4 and related areas have a similar implicit representation of the universe of natural images. The outcome of this work will be a new understanding of the functions of V2 and V4. These areas are of interest both because of their potential as a substrate for visual loss and recovery after brain damage, and because they are key parts of the processing chain by which visual signals are transformed to form decisions, guide actions, and create enduring memories of evanescent events.