CAREER: Efficient coding of visual,structural, and semantic scene information

Barnard College, New York NY

Investigators

Abstract

It is commonly said that “a picture is worth a thousand words”, a phrase that evokes the rich amount of information we can gain from the scenes that make up our visual world. But do all scenes contain the same amount of information? Intuitively, the answer seems to be ‘no’ -- we often encounter situations where we are overwhelmed with visual information, such as in a crowded concert venue or a cluttered desk. Further, when overwhelmed with visual information, we may make consequential mistakes, such as failing to find a tumor on a medical scan or crashing one’s car. This CAREER award aims to understand what types of scene information create overload and the time course of neural processing when overcoming information overload. Using both behavioral and electroencephalography (EEG) measures, we assess four levels of information, ranging from purely visual to semantic. These experiments provide insights into the mechanisms of visual perception and may enable designers to create spaces that minimally tax our cognitive resources. This award also takes meaningful steps toward democratizing training in basic computing. The PI and students work to create an open educational multi-media textbook that trains students in scientific computing skills. This CAREER award aims to gain insights into the mechanisms of scene perception by assessing the system under information overload. We gain insights into cognitive and neural mechanisms when we push systems to their limits. Rapid visual perception has intrigued researchers because the speed of perception places bounds on the types of neural mechanisms that can achieve recognition. However, most work centers around the successes of rapid scene understanding than its failures. This work assesses how four levels of increasing informational complexity (visual, object-based, semantic, and experiential) contribute to early scene processing. Specifically, the research tests how each information level affects performance in rapid scene detection and classification tasks and how each alters the time course of information processing using EEG. The results of these experiments reveal what types of information affect visual processing and at what time scales, providing critical insights into the mechanisms of rapid visual perception. The PI collaborates with students to create an open multimedia textbook on scientific computing skills that are often missing from early computer science classes. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.