Object, face, body and scene representations in the human brain

National Institute Of Mental Health

Investigators

Linked publications, trials & patents

Abstract

The goal of this research is to understand how we see what we see: how does the brain analyze the light falling on the retina of the eye to reveal a world full of people, places, and things? Recent work has focused on three major areas: 1) Understanding the multidimensional nature of objects Real-world, natural scenes are incredibly complex and heterogeneous, yet we are able to understand the nature of those scenes effortlessly – we can categorize them and identify objects and people within those scenes and plan actions toward individual elements. While prior studies have identified brain regions that appear to be specialized for processing faces, object and scenes, it remains unclear what the precise roles of these different regions are, what information they contain, and how broadly representations are distributed. One of the major challenges in understanding visual perception in the brain is the wide range of different objects and scenes that we experience. Over the last few years we have run an ongoing set of studies aimed at capturing the breadth of our understanding of the visual world, which we call the THINGS Initiative. Despite this diversity of our visual experience, studies often use a small number of hand selected object or scene categories, but it becomes unclear how representative of real-world processing the subsequent findings are. To overcome this challenge, we developed a large-scale database (THINGS) of 1,854 diverse types of objects sampled systematically from concrete picturable and nameable nouns in the American English language. The THINGS database provides a rich resource of object concepts and object images and offers a tool for both systematic and large-scale naturalistic research in the fields of psychology, neuroscience, and computer science. We used this database in large-scale behavioral experiments using online crowdsourcing, sampling over 4 million trials in more than 5,000 participants. Applying a computational model of the task, we were able to identify 66 core dimensions of our internal mental representations of objects, providing a comprehensive and fine-grained characterization of this object representational space. In addition to the behavioral data we also collected extensive functional MRI and MEG data while people viewed images from the THINGS database. The model we developed based on the previously collected behavioral responses (described above) provides a critical framework for investigating the underlying neural representations of objects. Analysis of the fMRI data revealed that the individual dimensions are represented broadly across visual cortex, including both early visual and high-level cortex, as well in parietal and frontal regions. Further, these behaviorally relevant dimensions were superior to object category information at predicting cortical brain responses. Analysis of the MEG data revealed that the individual dimensions are rapidly represented, starting around 80-100 ms, with different dimensions showing different temporal dynamics that may reflect the representation of visual versus conceptual features of the objects. Further, early responses tended to generalize across subjects, suggesting they are more stimulus-driven, while later responses showed less generalization, suggesting they reflect more subject-specific representations that might be linked to individual experience. Beyond exploratory studies as part of the THINGS Initiative, we have also probed highly specific questions. First, we investigated how physical experience with real objects affects the representation of images of those objects. Prior to entering the MRI scanner, participants handled a set of objects for 30 seconds each. Then, while they were in the scanner they were shown heterogeneous images of those same objects as well as matched set of images for unexplored objects. We found that exploration of the objects had no detectable impact on visual cortex, but led to much stronger activation in medial parietal regions that have often been linked to episodic memory and explicit memory tasks. This work highlights the importance of considering real world objects and not just images of novel objects for revealing how our understanding of the natural world develops. 2) Impact of eye movements on object processing To successfully navigate our environment, we move our eyes every ~250 ms to bring items of interest from our peripheral vision to central vision. With each eye-movement, our visual system receives a new perspective, so how does the brain cope with this continuous update of visual information? Here, we examine if information in our peripheral vision is maintained across an eye-movement to interact with neural processing directly after the eye-movement, demonstrating an extraordinary efficiency of the visual system. Specifically, participants were asked to make eye-movements while viewing images on a screen in an MRI scanner. During an eye-movement the image on the screen (face or building) could either 1) stay the same, or 2) change. When the image changed our ability to determine what the visual system was processing was degraded relative to when the image stayed the same. This indicates that there must have been transference of feature information from prior to the eye-movement which impacts processing after the eye-movement. This study demonstrates the visual system is maintaining information across eye-movements to facilitate processing after an eye-movement. Characterizing neurotypical processing across eye movements in the visual system is critical as it is hypothesized these mechanisms are atypical in clinical populations (such as individuals with Schizophrenia). Using neurotypical data, we can pinpoint exactly when there is divergence with atypical behavior and neural processing to facilitate eventual development of rehabilitation techniques. 3) Replicating the effects of non-invasive brain stimulation Non-invasive brain stimulation (NIBS) techniques have the potential to demonstrate the causal impact of targeted brain regions on specific behaviors. Transcranial random noise stimulation (tRNS) is one form of transcranial electric stimulation in which an alternating current is passed between electrodes at random frequencies. High-frequency tRNS (hf-tRNS) is thought to enhance excitability and have facilitatory effects on behavior, demonstrated in healthy and clinical populations. Due to the potential application of tRNS, clear demonstrations of the efficacy and replicability of stimulation are critical. Here, we focused on replicating the facilitatory effect of hf-tRNS over the human middle temporal complex (hMT+) on contralateral motion processing. In prior work, the improvement in performance was specific to global motion processing in the visual field opposite to stimulation. However, our results indicated that hf-tRNS does not improve motion discrimination with n = 42. Specifically, we were unable to replicate a global motion processing facilitation following hf-tRNS to hMT+. Although our lack of replication could be due to minor changes in the protocol from the original study, for hf-tRNS to become a widely applied method, its modulatory effect should be robust to slight adjustments to the procedure. Collectively, these studies provide important information about the cognitive processes supporting our understanding of the visual world. Elucidating how the brain enables us to recognize objects, scenes, faces and bodies provides important insights into the nature of our internal representations of the world around us. Understanding these representations is vital to identify and characterize the underlying deficits in many mental health and neurological disorders.