Object, face, body and scene representations in the human brain

National Institute Of Mental Health

Investigators

Linked publications, trials & patents

Abstract

The goal of this research is to understand how we see what we see: how does the brain analyze the light falling on the retina of the eye to reveal a world full of people, places, and things? One of the major areas of ongoing research focuses on understanding the multidimensional representations of objects (NCT00001360). Real-world scenes are incredibly complex and heterogeneous, yet we are able to categorize them and identify objects and people within those scenes effortlessly. While prior studies have identified brain regions that appear to be specialized for processing faces, object and scenes, it remains unclear what the precise roles of these different regions are and what information they contain. One of the major challenges in understanding visual perception in the brain is the wide range of different objects and scenes that we experience. Despite this breadth, studies often use a small number of hand selected object or scene categories, but it becomes unclear how representative of real-world processing the subsequent findings are. To overcome this challenge, we developed a large-scale database (THINGS) of 1,854 diverse types of object sampled systematically from concrete picturable and nameable nouns in the American English language. The THINGS database provides a rich resource of object concepts and object images and offers a tool for both systematic and large-scale naturalistic research in the fields of psychology, neuroscience, and computer science. We used this database in a large-scale behavioral experiment using online crowdsourcing, sampling 1.46 million trials in more than 5,000 participants. Using a computational model of the task, we were able to identify 49 core dimensions of our internal mental representations of objects, providing a comprehensive and fine-grained characterization of this object representational space. We have recently extended this work, collecting an additional 3 million trials of data, which expanded the number of dimensions to 66. In addition to the behavioral data we also collected extensive functional MRI and MEG data while people viewed images from the THINGS database. The model we developed based on the previously collected behavioral responses (described above) provides a critical framework for now investigating the underlying neural representations of objects. Analysis of the fMRI data reveals that the individual dimensions are represented broadly across visual cortex, including both early visual and high-level cortex, as well in parietal and frontal regions. Further, these behaviorally relevant dimensions were superior to object category information at predicting cortical brain responses. Analysis of the MEG data reveals that the individual dimensions are rapidly represented, starting around 80-100 ms, with different dimensions showing different temporal dynamics that may reflect the representation of visual versus conceptual features of the objects. To investigate memory for objects and determine what makes some images more memorable than others, we collected an additional dataset of more than one million recognition memory judgments. Using these data, we built a model of object features that is predictive of image memorability. We found that conceptual features exert a stronger influence than visual features on what we remember. Collectively, these studies of multidimensional object representations provide important insights into the cognitive processes supporting our understanding of objects. Elucidating how the brain enables us to recognize objects, scenes, faces and bodies provides important insights into the nature of our internal representations of the world around us. Understanding these representations is vital to identify and characterize the underlying deficits in many mental health and neurological disorders.