ARRA: Identifying Objects Within Scenes: Combining Context and Features in Visual Object Recognition

Florida Atlantic University, Boca Raton FL

Investigators

Abstract

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). Despite nearly half a century of research, the human ability to recognize objects visually remains a largely unsolved puzzle. Previous research on object recognition has primarily considered cases in which the target is viewed in isolation. However, the visual system can use contextual information -- such as the presence of other objects in the scene or knowledge about the kind of environment in which the object is found -- to determine the identity of an object as well. The contribution of this kind of information is especially clear when the image of the object itself is insufficient on its own. For example, a small yellow patch might be identified as a partially obscured banana in the context of a fruit bowl or as a leaf in the context of a tree. In an NSF-funded research project, Dr. Elan Barenholtz at Florida Atlantic University will use behavioral and computational techniques to examine two central questions regarding the role of context in object recognition: 1) How do people acquire knowledge about the relations between objects and their contextual scenes (for example, the likelihood of specific objects appearing in a certain type of context)? 2) How is this knowledge put to work in recognizing objects whose images have been degraded and cannot be recognized on their own? This research will employ two experimental methodologies: The first will use computer-generated artificial scenes in which participants must first learn the object/context relations from scratch and later use this knowledge in a recognition task. The second technique will test object recognition abilities in photographs of real world environments, including pictures of participants' own homes or workplaces. In this case, subjects will have knowledge about the expected object/context relations, based on their long-term experience, particularly when the environment is highly familiar to them. Human performance in these tasks will be assessed using statistical methods to assess the contribution of contextual information in object recognition. Understanding human visual object recognition holds great promise for brain science -- as much as a third of the human cortex is thought to be devoted to visual processing. Such understanding is also important for designing artificial vision systems, which carry an enormous array of potential applications. However, previous theoretical techniques, which focused on specialized algorithms for extracting 3-dimensional structure from individual objects, have proven largely unsuccessful. Dr. Barenholtz's research represents a strong departure from earlier approaches, as it assumes that visual recognition relies on inferential strategies that draw on an individual's broad knowledge about the world and his or her experience with specific environments. This approach treats vision as relying on similar tools as other cognitive processes, such as inference and decision-making, suggesting that there may be a great deal of previously unexplored common ground across these different disciplines. By putting the "cognition" back into "recognition," this research has the potential to contribute to some long awaited breakthroughs in the field of visual recognition.