CAREER: Integrated system for object and scene recognition

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Abstract Title: Integrated system for object and scene recognition PI: Antonio Torralba Institution: MIT In traditional computer vision, scene and object recognition are two related visual tasks generally studied separately. By devising systems that solve these tasks in an integrated fashion it is possible to build more efficient and robust recognition systems. At the lowest level, significant computational savings can be achieved if different categories share a common set of features. More importantly, jointly trained recognition systems can use similarities between object categories to their advantage by learning features which lead to better generalization. In complex natural scenes, object recognition systems can be further improved by using contextual knowledge both about the objects likely to be found in a given scene, and also the spatial relationships between those objects. Object detection and recognition is generally posed as a matching problem between the object representation and the image features while rejecting the background features using an outlier process. The PI will formulate object detection as a problem of aligning elements of the entire scene. The background, instead of being treated as a set of outliers will be used to guide the detection process. In developing integrated systems that try to recognize many objects, the lack of large annotated datasets becomes a major problem. The PI created and will extend two datasets; LabelMe and the 80 million tiny images datasets. LabelMe is an online annotation tool that allows sharing and labeling images for computer vision research. Both datasets offers an invaluable resource for research and teaching on computer vision and computer graphics. The datasets are also intended to foster creativity, as they allows students at all levels to explore well established algorithms as well as devise new applications in computer vision and computer graphics. The PI will also develop new image and video datasets by exploiting the millions of images available on the internet. The creation of robust systems for scene understanding will have a major impact on many fields by allowing the creation of smart devices able to interact and understand their environment, from aids to the visually-impaired, to autonomous vehicles, robotic assistants, or online tools for searching visual information. The PI will extend his teaching and research activities beyond the boundaries of the classroom and the laboratory by developing a substantial amount of online material. URL: http://people.csail.mit.edu/torralba/integratedSceneRecognition/