Neural Mechanisms of Natural Visual Perception

National Institute Of Mental Health

Investigators

Linked publications & trials

Abstract

Vision forms the foundation for much of our thinking, our memory, and our language. Our research program investigates the brain mechanisms underlying how we see objects, scenes, and individuals. Historically, the investigation of vision has derived from experiments in which stimuli are presented alone on a screen. This isolate presentation of simplified stimuli allows for the evaluation of how brain areas are specialized for certain types of visual patterns. This approach has led to a standard view of the visual brain that is hierarchical and modular. In this framework, a specific visual percept is assumed to emerge from the selective activation of some brain areas over others. An important question, however, is whether this framework extends to more natural modes of vision, where the visual environment is complex, immersive, and temporally continuous. In one project concluded in the last year (Khandhadhia 2023, PNAS), we investigated the 3D nature of vision as it pertains to the brain's encoding of objects. In contrast to most studies of object encoding, which emphasize the brains tolerance to different spatial transformations, we demonstrate that object-selective neurons are highly sensitive to natural visual geometry. Our findings demonstrate that the size variable affecting the tuning object selective neurons is not the angle cast on the retina, but instead the absolute size of objects in centimeters. This finding has implications for the nature of visual representations in the brain. We also performed a follow-up study investigating the responses of neurons within the so-called face patch system to movies consisted of natural behaviors, such as socially interacting and eating (Park et al, 2022, Sci Adv). By applying a novel approach, in which we mapped the responses if individual face-selective neurons to fMRI signals across the brain, we showed that the face patch network is, in fact, pervaded by multiple, parallel subnetworks. Thus, the apparent functional principles of the face system appear entirely different when experiments depart from the simple presentation of isolated images on the screen. In a related study, we investigated whether there is a fundamental difference between continual visual experience, as for example experienced while watching a movie, and the discrete stimulation of the brain that forms the basis of most visual neuroscience experiments (Russ et al 2023, Neuron). In our experiments, we have allowed subjects to freely view natural movies that play out over several minutes, record their gaze and brain activity during this time. This study, revealed that the temporal context critically shapes how visual details are processed. For example, movie content presented discretely in brief snippets and out of order elicits wholly different responses from the brain compared to the same content presented in its original continuous movie context. At present, we are pursuing experiments inside the immersive dome environment, where it is possible to examine aspects of visual brain activity that are hidden when animals view stimuli on a small screen. Given our recent results, we anticipate that the spatial contribution to object representations has been severely underestimated in the past, where researchers have often treated object representations almost symbolically and devoid of spatial context. In the next years, we hope to understand this important aspect of vision, which can give new insights how the human brain approaches natural modes of visual perception and behavior.