CAREER: Situated Recognition: Learning to understand our local visual environment

University Of North Carolina At Chapel Hill, Chapel Hill NC

Investigators

Abstract

This project develops computer vision technologies for recognizing objects in our daily lives. For recognizing visual content around us, where cameras can record multiple images over a period of time, there is an opportunity to take advantage of context that is not available for internet images. This project pursues new representations and computational strategies exploiting this context efficiently to achieve high-quality visual recognition in our environment. Balanced against the opportunity of using this context is the challenge of making recognition work in any particular environment, in the face of clutter, occlusion, non-canonical views, and idiosyncratic appearance variation. The methods developed can be a core part of developing technology to help computer vision systems scale to recognize everything in our daily world. The research leads to automated systems for better understanding and monitoring of our daily environment, improved human-computer interaction, and encourages more research in this area. This research direction is different from the majority of work in recognition that has focused on internet images collected from the web. The biases of such web-collected may lead to models that do not generalize to a particular environment. Situated recognition allows exploiting local context, including human interaction and spoken language, to build models specific to an environment and furthermore to the parts of an environment that are important to people. The project collects multiple datasets stressing multi-view imagery and long-term observation of environments while sampling a wide variety of settings. The research team develops algorithms to parse and detect objects by exploiting context, efficient re-use, and context-dependent saliency; and uses situated natural language to drive automatic learning of visual recognition models. Project Webpage: http://acberg.com