Joint Space-Time Analysis and Characterization of Image Sequences

Trustees Of Boston University, Boston

Investigators

Abstract

Janusz Konrad Boston University This research is concerned with the processing of visual information captured by a video camera. To date, image sequences have been typically analyzed and processed in groups of two frames; by differentiating one frame from another, short-term image dynamics can be measured, such as changed areas (e.g., occlusions) or pixel displacements. Although the approach has been very successful to date (e.g., MPEG compression standards), further gains are difficult to attain based on two image frames only. This research project offers a different approach to the analysis of image sequences - one based on a joint processing of multiple image frames at a time. This joint treatment of, for example, 10 or 20 frames is expected to result in new gains in video compression, more reliable video database querying, and more accurate detection of innovations (occlusion and exposure areas) that are of interest in surveillance applications. The primary problem attacked in the project is the joint space-time segmentation of an image sequence into "object tunnels", i.e., 3-D volumes carved out in the space of horizontal, vertical and temporal coordinates by a moving object. The estimation of object tunnels is approached as a volume competition problem, and solved using active-surface evolution equations embedded into the level-set solution framework. In order to model motion of points within each object tunnel, a new spatially-parametric, temporally-quadratic motion model is studied. Various cost functionals relating object-tunnel intensities to the underlying motion are investigated. Since the standard volume competition can only extract a single moving object from background, an extension to more objects by means of multiple "repelling" surfaces evolving simultaneously is studied as well.