CAREER: Generalized Image Understanding with Probabilistic Ontologies and Dynamic Adaptive Graph Hierarchies

Suny At Buffalo, Amherst NY

Investigators

Abstract

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). From representation to learning to inference, effective use of high-level semantic knowledge in computer vision remains a challenge in bridging the signal-symbol gap. This research investigates the role of semantics in visual inference through the generalized image understanding problem: to automatically detect, localize, segment, and recognize the core high-level elements and how they interact in an image, and provide a parsimonious semantic description of the image. Specifically, this research examines a unified methodology that integrates low- (e.g., pixels and features), mid- (e.g. latent structure), and high-level (e.g., semantics) elements for visual inference. Adaptive graph hierarchies induced directly from the images provide the core mathematical representation. A statistical interpretation of affinities between neighboring pixels and regions in the image drives this induction. Latent elements and structure are captured with multilevel Markov networks. A probabilistic ontology represents the core knowledge and uncertainty of the inferred structure and guides the ultimate semantic interpretation of the image. At each level, rigorous methods from computer science and statistics are connected to and combined with formal semantic methods from philosophy. A symbiotic education plan involving graduate and undergraduate mentoring and education, professional tutorial courses at the boundary of vision and ontology, and K-12 outreach is incorporated into the research plan. The research and education, disseminated broadly through both the applied science and semantics/philosophy literatures, lays a foundation on which to both utilize and automatically extract rich semantic information from images and other signal data for critical application areas such as internet vision, autonomous navigation, and ambient biometrics.