Mining and Indexing Spatio-Temporal Patterns in Video Databases of Human Motion

Trustees Of Boston University, Boston

Investigators

Abstract

The aim of this research project is to develop and test methods for indexing, retrieval, and data mining of human motion trajectories in video databases. Computer vision techniques are being devised for automatic extraction of human motion time series data from video. Data mining algorithms are being developed that can be used to discover clusters and other patterns in the extracted motion time-series data. One promising direction being explored is to model the observed motion time series sequences with a finite mixture of Hidden Markov Models (HMMs). Use of the HMM representation presents certain advantages with regard to modeling; however, it presents important challenges for the design of efficient clustering, indexing, and retrieval algorithms. Thus more efficient, sampling-based and embedding-based methods must be formulated. The resulting ideas are evaluated in a prototype video retrieval system, with real-world video datasets that depict human body motion. Synthetic sequences (e.g., generated via computer graphics) are used in quantitative performance experiments where ground truth information is required. The products of this research effort can enable numerous applications that are valuable to society: homeland security; video-based analysis of human biomechanics for occupational safety, as well as dance and sports training; archive management and analysis for news, entertainment, and sports video; and video database management for non-intrusive monitoring of the motion patterns of handicapped, infirm, or elderly people to detect decline, danger, and to alert caregivers when needed. Results can be accessed at the project's Web site (http://www.cs.bu.edu/groups/ivc/MotionMining/).