Neural Basis of Shape from Texture

State College Of Optometry, New York NY

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): People often need to judge the shapes and movements of 3-D objects from a distance. Images on the retinae are 2-D, but pattern and motion information contain cues about 3-D shapes. Building on our previous work we expect to make significant progress in understanding 3-D shape perception from these cues, and thus offer a prototype for how the brain extracts information from the world to infer environmental properties. When surfaces have texture patterns, deformations of these patterns in retinal images provide clues for the brain to judge 3-D shape. When signals from the eyes reach the first cortical area V1, they are processed by neurons that are selectively tuned to orientations and spatial frequencies. We parsed texture deformations into orientation flows and spatial frequency gradients, to show that particular orientation flows evoke percepts of specific 3-D shapes, whereas frequency gradients provide cues to relative depth. These results led to models of how later cortical neurons could extract texture patterns and signal 3-D shapes. We now propose to extend our approach beyond static objects. When objects change shape (e.g. by bending, coiling, or contracting) as they move (e.g. walk, tumble, crawl, hop, slide, or swim), changes in the retinal image create patterns of local velocities that provide additional cues to 3-D shape. Since any retinal image can result from projections of many different 3-D objects, the brain relies on prior assumptions to infer the correct shape. Previous studies have only looked at rigid objects and used shape inference models based either on the brain assuming that the object is rigid or that faster points are nearer. We will use novel stimuli that put these prior assumptions in conflict, and thus examine how the brain potentiates a prior. This section will culminate in a model for choosing between conflicting assumptions, something that is often required in perception and cognition. Next, we will use randomly deforming non-rigid 3-D waves to examine how global shape properties, e.g. symmetry, influence perceived object motions by selectively combining disparate outputs of motion sensitive neurons. These results will unveil interactions between the form and motion cortical systems. Finally, we will examine observers' percepts of dynamic shape changes that require more sophisticated analyses of retinal velocity patterns. Neurons in the motion sensitive cortical area MT respond to 1-D motion shear and compression/divergence, so we will extract these qualities and combine them into 2-D velocity patterns of divergence, rotation, and deformation. Neural filters formed by these patterns will be used to explain perceived changes in 3-D shapes. We will base our filters on responses of MT and later cortical neurons to our stimuli, measured in a parallel project. We will thus present the first neural model that can explain observers' percepts of both rigid and non-rigid textured objects. The performance of our model will be compared against the best computer-vision models on motion-capture data from real deforming objects. We expect this project to introduce new ideas, methods and results for understanding visual perception of 3-D shapes and its deficits in neurological patients.