RI: Integrating Illumination, Motion and Shape Models for Video Analysis

University Of California-Riverside, Riverside CA

Investigators

Abstract

Proposal 0712253 PI: Amit Roy Chowdhury Institution: University of California - Riverside Co-PI: Gopi Meenakshisundaram Institution: University of California - Irvine Title: Integrating Illumination, Motion and Shape Models for Video Analysis This project involves a fundamental research program in video analysis that is built upon a novel framework for integrating different independent modalities of image formation - illumination conditions, object shape, motion and surface reflectance. We focus on the problem of pose and illumination invariant object recognition in video, based on learned models of lighting, motion and shape. We also develop novel scene relighting methods using the illumination models learned from natural videos. The proposed research proceeds by first developing a mathematical framework that relates the appearance of a video sequence with the lighting conditions, motion of the objects being imaged, and their shape and surface properties. Thereafter, it addresses the inverse problem of recognition from video. The overall mathematical approach is to combine precise geometrical models with statistical data analysis tools, thus combining accuracy and robustness. This research will benefit a large number of existing applications and create new ones that rely on efficient object tracking under changing environmental conditions. They include applications in national priority areas like homeland security, monitoring of nuclear installations and border security, in commercial interests like video communications, multimedia databases and entertainment, in social causes like wildlife and environmental monitoring, and in medical and biological applications relying on video analysis. Scene relighting by learning the motion and illumination of objects from natural videos will also have a significant impact in creation of realistic virtual environments. Progress on this project will be regularly updated at http://www.ee.ucr.edu/~amitrc/JMIS.htm