US-German Collaboration: Integration of Bottom-Up and Top-Down Signals in Visual Recognition

Children'S Hospital Corporation, Boston MA

Investigators

Abstract

Visual cognition is orchestrated by the interaction of 'bottom-up' (feed-forward) processes that carry sensory information and 'top-down' (feed-back) processes that modulate the incoming input in the context of goals, tasks, emotions and stored information. At the anatomical level, each area within the cerebral cortex is heavily innervated by both feed-forward signals and feed-back signals. With funding from the National Science Foundation, Gabriel Kreiman, Ph.D. of Children's Hospital Corporation (Boston, Massachusetts) in collaboration with Andreas Schulze-Bonhage, Ph.D., of the Freiburg University Hospital (Freiburg, Germany), is investigating the dynamical integration of bottom-up and top-down neural signals, by combining computational models and machine learning techniques for data analysis with high-resolution neurophysiological recordings from the human temporal lobe. Researchers have long recognized that top-down and bottom-up signals play a key role in visual recognition, however, the relative contribution and interactions between these signals remain unclear. The research project is focused on a particular aspect of cognition, namely our ability to visually recognize patterns, which is central to most everyday tasks. Even the best machine computational models available today only provide a coarse approximation to the complex neurophysiological responses found in higher visual cortex. Not surprisingly, a three-year-old can outperform sophisticated computational algorithms in recognition tasks, such as navigation in complex environments or recognizing objects in cluttered scenes. The research project focuses on three progressively more complex tasks that rely increasingly on top-down influences. The first research aim involves top-down influences during recognition of objects in a cluttered visual stimulus. The second aim examines whether neurophysiological responses in the human temporal lobe can support recognition from partial object information. This question is being approached through studying the phenomenon of object completion. The third aim combines visual stimulus clutter and occlusion in a complex realistic recognition scenario. For this aim, the researchers are examining the influences of attention and task-related goals on neurophysiological activity while epilepsy patients play a custom-designed video game. These neurophysiological data take advantage of the rare opportunity to combine high-resolution neurophysiology, computational models, and behaviorally complex tasks to carry out research that would be difficult with non-human animals. By furthering the understanding of the transformation of perceptual information into cognition, the researchers are contributing to two broader goals: The goal to help alleviate the challenging conditions involved in cognitive disorders through the development of interfaces between brains and machines, and the goal to apply knowledge about neuronal circuits to develop computational algorithms that automatically extract cognitive information from sensory data. Building a fast, robust, and reliable artificial vision system would have profound repercussions in many areas of science and engineering, including pattern recognition, surveillance and security, automatic navigation and clinical image analysis. These scientific and engineering advances could in turn translate into important real-world applications of interest for industrial partnerships. Understanding the visual system relies on many skills ranging from computer science to physics, neuroscience, and psychology. The research efforts are complemented by educational and outreach initiatives aimed at training interdisciplinary scientists. The training is producing multidisciplinary students who can build on their fundamental scientific skills and apply this knowledge to challenging clinical and engineering problems. This project is jointly funded by the Cognitive Neuroscience Program, the Social Behavioral and Economics Division, Collaborative Research in Computational Neuroscience, and the Office of International Science and Engineering. A companion project is being funded by the German Ministry of Education and Research (BMBF).