Object, face, body and scene representations in the human brain

National Institute Of Mental Health

Investigators

Linked publications & trials

Abstract

The goal of this research is to understand how we see what we see: how does the brain analyze the light falling on the retina of the eye to reveal a world full of objects, people and things? Ongoing work focuses on two main areas of research: i) understanding the multidimensional representations of objects, and ii) determining the nature of object representation during occlusion (NCT00001360). i) Understanding the multidimensional representations of objects Real-world scenes are incredibly complex and heterogeneous, yet we are able to categorize them and identify objects and people within those scenes effortlessly. While prior studies have identified brain regions that appear to be specialized for processing faces, object and scenes, it remains unclear what the precise roles of these different regions are and what information they contain. One of the major challenges in understanding visual perception in the brain is the wide range of different objects and scenes that we experience. Despite this breadth, studies often focus on a small number of hand selected object or scene categories, but it becomes unclear how representative any subsequent findings are. To overcome this challenge we developed a large-scale database (THINGS) of 1,854 diverse types of object sampled systematically from concrete picturable and nameable nouns in the American English language. This database provides a rich resource of object concepts and object images and offers a tool for both systematic and large-scale naturalistic research in the fields of psychology, neuroscience, and computer science. We used this database in a large-scale behavioral experiment using online crowdsourcing, sampling 1.46 million trials in more than 5,000 participants. Using a computational model of the task, we were able to identify 49 core dimensions of our internal mental representations of objects, providing a comprehensive and fine-grained characterization of this representational space. In addition to the behavioral data we also collected extensive fMRI and MEG data on images from the THINGS database. The work described above provides a critical framework for now investigating the underlying neural representations of objects. We are establishing how these core dimensions are reflected in the neural signals over both space (fMRI) and time (MEG). Our analyses reveal highly distributed representations of the dimensions that include both high level and early visual cortex. Further, the temporal dynamics show wide variation in the evolution of responses that may reflect the differential contribution of visual and semantic information. These results provide important insights into the cognitive processes supporting our understanding of objects. ii) Determining the nature of object representation during occlusion Visual information is interrupted frequently due to occlusion, eyeblinks, and saccades. However, objects in the visual environment seem to persist through these perceptual gaps. We have been examining the nature of object representations during perceptual gaps caused by occlusion. First we used MEG to track how object representations unfold over time: before, during and after occlusion. Focusing on color, shape and position as different object features, when the moving objects were fully visible, there was some evidence for information about all features in the MEG signal. During occlusion, while some position information persisted, we could find no evidence for representation of other features. These results challenge the notion of a perception-like representation of moving objects during occlusion and open up new questions about how the visual system overcomes perceptual gaps to support the perception of a meaningful, continuous stream of information. Second, we conducted a series of behavioral experiments to further probe the nature of representations during occlusion. Participants were asked to report on the position of an object undergoing occlusion or on the time when the object would reappear from behind the occluder. While participants were accurate in estimating position during the early part of occlusion, their performance quickly deteriorated, suggesting a limited representation of occluded objects consistent with the MEG data. However, participants were very accurate in determining whether an object re-emerged from behind the occluder at the correct time or not, suggesting that integration between the pre- and post-occlusion periods might be critical for understanding occlusion events and might underlie our sense of object persistence. Elucidating how the brain enables us to recognize objects, scenes, faces and bodies provides important insights into the nature of our internal representations of the world around us. Understanding these representations is vital in trying to determine the underlying deficits in many mental health and neurological disorders.