Object, face, body and scene representations in the human brain

National Institute Of Mental Health

Investigators

Linked publications, trials & patents

Abstract

The goal of this research is to understand how we see what we see: how does the brain analyze the light falling on the retina of the eye to reveal a world full of objects, people and things? During the past year we have continued to focus on both (i) perception and (ii) memory of complex visual stimuli, in particular real-world visual scenes, objects and people (NCT00001360). Perception: Real-world scenes are incredibly complex and heterogeneous, yet we are able to categorize them and identify objects and people within those scenes effortlessly. While prior studies have identified brain regions that appear to be specialized for processing faces, object and scenes, it remains unclear what the precise roles of these different regions are and what information they contain. One of the major challenges in understanding visual perception in the brain is the wide range of different objects and scenes that we experience. Despite this breadth, studies often focus on a small number of hand selected object or scene categories, but it becomes unclear how representative any subsequent findings are. To overcome this challenge we developed a large-scale database (THINGS) of 1,854 diverse types of object sampled systematically from concrete picturable and nameable nouns in the American English language. This database provides a rich resource of object concepts and object images and offers a tool for both systematic and large-scale naturalistic research in the fields of psychology, neuroscience, and computer science. We used this database in a large-scale behavioral experiment using online crowdsourcing, sampling 1.46 million trials in more than 5,000 participants. Using a computational model of the task, we were able to identify 49 core dimensions of our internal mental representations of objects, providing a comprehensive and fine-grained characterization of this representational space. This work provides a critical framework for investigating the underlying neural representations of objects. We have now collected extensive fMRI and MEG data on images from the THINGS database. Our ongoing analyses aim to investigate how the fundamental dimensions identified in behavior are reflected in brain activity over both space and time. These results provide important insights into the cognitive processes supporting our understanding of objects. Memory: To understand the nature of object and scene memory, we have focused on two specific issues: 1) Relationship between activity elicited during perception and memory During memory recall, cortical reinstatement is thought to occur as an echoing of the neural patterns elicited during encoding. However, the precise information represented in these recall traces is relatively unknown, with previous work primarily investigating either broad distinctions (e.g. scenes versus objects) or specific individual images (e.g. exemplars of chair, hat), rarely bridging these levels of information. Here, we conducted an in-depth comparison of encoding and recall along a spectrum of granularity, from a coarse level (scenes, objects) to a mid level (e.g., natural, manmade scenes) to a fine level of individual categories (e.g., living room, cupcake). We found that there is decodable information at all levels of granularity in category-selective visual cortex. However, information during recall was primarily at the coarse level, with fine level information in some areas; there was no evidence of mid level information, suggesting a different representational structure to that during encoding. Further, we found some degree of segregation between those voxels showing the strongest effects during encoding and those during visual recall with the peaks of encoding-recall similarity in regions anterior to category-selective cortex. Collectively, these results demonstrate that while there are some similarities between representations during perception and recall, as highlighted in many prior studies, there are also clear differences that need to be elaborated. 2) Elucidating the content of scene memory When we recall a previously experienced event, what exactly are we remembering? Are our memories a precise, high-definition recording of that event, a low-resolution gist of that memory, or even just a verbal description of what we saw? Answering this question is an essential component of being able to tease apart the mechanisms of memory: what information is encoded and maintained, how memory decays over time, and what information is retrieved from these memories. In a series of studies, we have been investigating the content of memory for real world scenes using a drawing task. Participants study scenes and then after a delay, draw those scenes in as much detail as possible from memory. The resulting memory-based drawings are scored by thousands of online observers, revealing numerous objects, few memory intrusions, and precise spatial information. We have studied both young volunteers as well as people who claim to have no visual imagery (termed aphantasia). Further, we have investigated a common distortion that occurs in memory in which the boundary of the scene is either extended or contracted. Our findings show that not only is it possible to quantify the content of memory during free recall, but that those memories contain detailed representations of our visual experiences (although memory distortions may still occur). Aphantasic participants recalled significantly fewer objects than matched controls, with less color in their drawings, and an increased reliance on verbal scaffolding. However, aphantasic participants showed high spatial accuracy equivalent to controls, and made significantly fewer memory errors. These differences between groups only manifested during recall, with no differences between groups during the matched perceptual condition. This object-specific memory impairment in individuals with aphantasia provides evidence for separate systems in memory that support object versus spatial information. Elucidating how the brain enables us to recognize objects, scenes, faces and bodies provides important insights into the nature of our internal representations of the world around us. Understanding these representations is vital in trying to determine the underlying deficits in many mental health and neurological disorders.