CIF: Small: Collaborative Research: Geometry-aware and data-adaptive signal processing for resource constrained activity analysis

Arizona State University, Scottsdale AZ

Investigators

Abstract

The task of characterizing human activities using multimodal sensing and automated inference techniques is critically important in many applications. Human activities when observed using traditional audio and video sensors, as well as novel emerging sensors such as depth cameras, orientation sensors, and smart personal devices, result in complex, high-dimensional spatiotemporal signatures that are difficult to analyze. The sources of difficulty are many: high data throughput, the need for sensor registration, variable execution rates associated with activities, pose variability in sensing, the non-Euclidean nature of feature spaces due to physical constraints on environments and actors, and data corruption due to occlusions. The current techniques, involving Euclidean representations and multivariate analyses, fall short in characterizing human activities, as they do not handle non-Euclidean structures nor obtain invariances to pose and execution rates. This research uses tools spanning differential geometry, statistics and signal approximation theory to develop novel frameworks for characterizing human activities. These fundamental tools lead to comprehensive solutions that are applicable to a broad swath of traditional and emerging sensors. The salient aspects of this approach are: 1) geometry awareness, encompassing both classical Euclidean as well as non-Euclidean feature spaces, 2) invariance to sensor placement and execution rate tightly integrated into the representation, and 3) data adaptivity leading to low bitrate representation of human activities for reduced communication and low computational scenarios. Applications of this research include monitoring of human activities using off-the-shelf sensors in resource-constrained environments, such as at homes and on mobile devices.