EAGER: Smart Space-Time Sampling for Recovering and Recognizing Dynamic Scenes

Rochester Institute Of Tech, Rochester NY

Investigators

Abstract

Traditional, dynamic scenes are captured as video frames sampled at regular space-time grids. For many computer vision tasks, however, this uniform sampling may be either inefficient (e.g., low light, high-speed motion) or unnecessary (e.g., motion/change/event detection). This project explores non-uniform, adaptive sampling schemes that exploit the underlying structures of space-time volumes (e.g., sparsity, temporal coherence, statistical priors). These sampling schemes are implemented with novel programmable pixel-wised coded exposure and aperture in cameras. The captured information-rich coded projections of space-time volumes are used for video reconstruction or directly as features for motion/event detection. In addition to higher efficiency in imaging and higher signal-to-noise ratio in reconstructed results, the method also provides benefits in data security and privacy protection for video surveillance because decoding the captured images requires the knowledge of coded patterns and dictionaries. This research has many applications in surveillance, machine vision inspection, and high-speed imaging. The developed technology is being tested in transportation imaging for traffic monitoring and accident detection. A database of high-speed videos of traffic scenes and events is being captured and plan to be released online when it is finished. In addition to videos, the technical approach can also be applicable to other high-dimensional signals such as light fields or light transport matrices.