EAGER: Diverse M-Best Predictions from Probabilistic Models

Virginia Polytechnic Institute And State University, Blacksburg VA

Investigators

Abstract

Computer Vision systems must deal with significant levels of ambiguity - from inter- and intra-object occlusion and varying appearance, lighting, and pose. Probabilistic models provide a principled framework for dealing with uncertainty and for converting evidence into a posteriori belief about the world. Typically, a vision system uses this belief to predict the "most likely" or maximum a-posteriori hypothesis. Unfortunately, our current models are inaccurate and this single-best hypothesis is often incorrect. This project explores a novel way to allow vision systems to hedge against uncertainty by producing multiple plausible hypotheses. Specifically, this project develops techniques for finding a diverse set of high-probability solutions from probabilistic models. The project focuses on (a) interactive object cutout (where multiple segmentations are shown to the user to expedite convergence to an acceptable result); (b) semantic segmentation (where multiple plausible scene labelings are propagated to subsequent stages of a cascade for higher-order processing); (c) person/object tracking (where multiple localization hypotheses on each frame reduce the search space of a sequence tracker). This project is producing new scientific knowledge in the context of probabilistic reasoning and advancing the state of art in computer vision. The techniques developed are useful for other AI domains such as Speech and Natural Language Processing. The PI and his students are broadly disseminating produced work by organizing workshops, tutorials, and journal special issues, and publicly sharing code and results. The project is engaging undergraduate students and women in computer science research.