CAREER: Identifying Spatial and Dynamical Patterns from Images

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

The quantitative modeling and automatic extraction of semantic information (objects, actions, and events) from images or videos has traditionally been a difficult and intriguing problem for scientists and engineers who are interested in system automation, human machine interface, machine intelligence, and information technologies. The basis for this difficulty, and interest, is largely attributed to physical, spatial, and dynamical complexity of visual patterns (of a shape or a process): high dimensionality and inherent variability in photometry, geometry, and dynamics are just a few of such characteristics. The proposed research program aims to tackle this complexity and variability by exploiting invariant properties in spatial and dynamical patterns that can be modeled as a hierarchy of discrete and continuous symmetries in the geometry, dynamics, and physics of visual patterns commonly encountered in an urban environment. Domain knowledge from photometry, multiple-view geometry, and systems theory will be used to analytically model and study both spatial and dynamical symmetries encoded in the visual data, and efficient numerical algorithms will be developed to detect such symmetries directly from images. High-level semantics of the images or videos can therefore be identified, inferred, or learned by machines more efficiently at the level of symmetries. We envision that such a modeling paradigm will marry the benefits of both analytical modeling and statistical inference techniques and significantly reduce the complexity in modeling, analysis, and computation. It will be the key to the success of developing efficient, accurate, and robust vision systems for identifying a wide range of objects, actions, and events in an urban environment. Direct outcome of the proposed research program will be scalable algorithms that can automatically generate three-dimensional geometric models from images or efficient systems that can identify in human actions and events from a large array of video input. Such algorithms will greatly facilitate applications such as security surveillance, traffic/environmental monitoring, automatic mapping of urban areas, vision-based navigation and coordination of autonomous robots, instant sports coverage/broadcast, movie edition/video indexing, and medical image analysis. Along with an increasing interest in the relations between symmetry and perception in cognitive science and psychology, the results from this program will also help provide an analytical and computational basis for the scientific study of biological and artificial visual perception in general. On the education end, the proposed research program provides unique opportunities for the development of new interdisciplinary courses that integrate scientific methods, mathematical skills, computational techniques, and laboratory experiments across multiple scientific and engineering disciplines, including computer vision, systems theory, and machine learning. These courses will help transform and enhance future college engineering education in robotics, machine vision/learning, and image processing. The illustrative examples and computer programs to be developed can also help teach many basic and important geometric concepts to high school students or to the general public through association with common visual experience and phenomena.