Mathematical Analysis of the Compositional Structure of Images

Brown University, Providence RI

Investigators

Abstract

The investigator and his colleagues study mathematical, statistical, and computational questions motivated by the problem in computer vision of defining and computing "structural scene description," descriptions of the objects in a scene and their relations to each other. The first goal is to obtain a theoretical understanding of the problem of inferring objects and their compositional structure, both in standard optical images and in laser range imagery and motion images. The second goal is to test this understanding by developing effective algorithmsfor the statistical inference of this structure, algorithms that work with real data at reasonable speed on current computers. The general approach is to formulate mathematical representations of patterns, typically compositional with a hierarchy of structures, to encode the variability of these structures in stochastic models, and to use the models to infer information about the image. Standard stochastic models are rarely adequate, prompting the development of new classes of probability measures or completely new directions for traditional models. Additionally, in the context of this application the team aims to develop mathematical, statistical, and computational ideas that are of broader use. For example, compositional issues in vision are similar to ones in the grammars of language. One aspect of the project studies this connection. The investigator and his five colleagues continue their mathematical, statistical, and computational investigations of a range of problems motivated by image processing and computer vision. The underlying question is simple enough: Here's an image, what is it an image of? Despite its simplicity, this is a hard question to answer. The approach taken here is to decompose the image into different components in some hierarchical structure and to use statistical analysis to infer information about the image from the relationships of the components within the structure. There are similarities with the grammatical structure of language, which the project explores. Recognition of objects in an image is a fundamental problem for both computer systems and biological systems. Advances are important for engineers developing computer vision applications, computer scientists seeking efficient algorithms in problems related to intelligent behavior of machines, and cognitive scientists studying human vision and language skills. Results of the project are published on CD-ROM, allowing demonstration of the dynamic behavior of algorithms in ways impossible through traditional publication modes and offering new ways to exploit multi-media communications for both education and research purposes.