RI: Small: Hierarchical Visual Scene Understanding

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Intelligent systems, both artificial and biological, must find effective ways to organize a complex visual world. The cross-disciplinary field of scene understanding is in need of a comprehensive framework in which to integrate cognitive, computational and neural approaches to the organization of knowledge. This research program aims to create a framework for organizing knowledge of visual environments that human and artificial systems encounter when navigating in the world or browsing visual databases. The aim is to determine which taxonomies are best suited for solving different visual tasks, and use computer vision algorithms to organize visual environments as humans do. For example, semantic relationships between scenes are well captured by a hierarchical tree (e.g. a basilica is a type of church, which is a type of building) but functional similarities between different environments may be best represented as clusters (e.g. restaurants, kitchens and picnic areas clustered as places to eat; offices and internet cafés as places to work). Because hierarchies and taxonomies provide a way of formalizing many types of contextual information (spatial, temporal, and semantic), they can be used to enhance the performance of computer vision systems at object and scene recognition, and aid in the development of smarter image search algorithms. Besides serving as a unified benchmark for comparing different models and theories, this enterprise offers new teaching and applied tools for research and courses, which will be made available through websites and symposia.