RI: Medium: Active Scene Interpretation by Entropy Pursuit

Johns Hopkins University, Baltimore MD

Investigators

Abstract

This project develops a new strategy for scene interpretation, especially for annotating cluttered scenes with instances from many object categories (e.g., a kitchen scene) and videos of people interacting with objects in everyday life (e.g., cooking). The research team develops a statistical model for scene interpretations and image measurements. One component of the model is a prior distribution on a huge interpretation vector. Each bit of this vector represents a high-level scene attribute with widely varying degrees of specificity and resolution ? some are very coarse (general hypotheses) and some are very fine (specific hypotheses). The other component is a simple conditional data model for a corresponding family of learned binary classifiers, one per bit. The scene interpretation is then computed by assessing hypotheses in a highly coarse-to-fine manner, using an image parsing algorithm called ?entropy pursuit? based on stepwise uncertainty reduction, and classifiers for detecting events in spatiotemporal volumes which leverage on recent advances at the intersection of machine learning and dynamical systems. The computational models and scene parsing algorithms developed in this project are broadly applicable to scene interpretation problems arising in many areas of science and engineering. Specific applications include home surveillance and security, assisted home living, infant and elderly care, etc. The project also provides research opportunities for graduate students in underrepresented minorities and even high school students.