Nonlinear Spatiotemporal Models for Decomposing Style Variations using Kernel Methods

Rutgers University New Brunswick, New Brunswick NJ

Investigators

Abstract

Robotics and Computer Vision Program ABSTRACT Proposal #: 0328991 Title: Nonlinear Spatiotemporal Models for Decomposing Style Variations using Kernel Methods PI: Elgammal, Ahmed Rutgers Univ New Brunswick The ultimate goal of this research is to model the changes in human appearance (shape and intensity) due to action being performed through generative spatiotemporal models that explicitly decomposes the variations due to the effects of personalized style (spatial and temporal) of the human performing the action. Learning such models would facilitate a unified framework for simultaneously solving of certain human motion analysis problem including: 1) providing spatiotemporal priors for tracking. Since the effect of personal style is explicitly decomposed, these priors can be specialized to the particular tracked subject. 2) parameterizing the personalized style effect in a way that will be useful for identifying the subject performing the action. 3) Detecting spatiotemporal action outliers. Inline with this ultimate goal, this proposal addresses the effect of style variations on appearance changes in terms of shape, i.e., human silhouettes. Human silhouette (shape) deformation is considered as a global form of appearance changes that carries sufficient information that can be exploited for further analysis of human motion. The observed human silhouette at each time instant is considered as a shape derived from a generic spatiotemporal model that can be specialized to the particular human being tracked through explicit decomposition of orthogonal style variation modes. Since the silhouettes undergo topological changes over time, correspondences between landmarks (features) are not always feasible. Therefore, the research will focus on global shape representations that do not require establishing feature correspondences. The research will focus on the use of explicit and kernel-based implicit nonlinear mapping approaches where the mapped silhouettes can be decomposed using multi-linear tensor decomposition into orthogonal factors given bases for each factor affecting the shape deformation such as body pose, spatial style, and temporal style.