CGV: EAGER: Simulation-Based Manipulation Capture for Dexterous Character Animation

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Through our hands, we communicate, care for ourselves and others, and use tools to affect our world. However, our understanding of how we control our hands to accomplish such feats is still in its infancy. Dexterous manipulation is especially challenging to study, as even "simple" manipulation tasks are complex in their details. The task of lifting a wrench into the hand, for example, can be decomposed into five separate actions, each of which may require a specialized control strategy. It is important to study human examples of such activities; through understanding human expertise we can reach practical outcomes such as physically intelligent animated characters for training and remote communication, robots capable of unprecedented dexterity, and prosthetic designs that far exceed the current state of the art in their elegance and functionality. The key hurdle in making a significant advance in these areas is the difficulty of capturing human manipulation in a form that facilitates analysis and study. The rapid sequence of contact events, the hand's large number of degrees of freedom, and close contact between the hand and object all contribute to creating an impossible capture task using traditional methods. The investigators are developing an alternative: simulation motion capture, where a user interacts with and guides a running simulation. Through this innovative approach, details such as contact timing, contact area, and contact forces are made available for the first time for general manipulation tasks. Key innovations include a fast simulation system for a deformable human hand (or full body) and novel techniques to control such a high degree of freedom simulation with intent and precision. In parallel, the investigators create a database of manipulation tasks, study new languages for action segmentation and control law development, develop robust autonomous controllers for grasping and manipulation, and study novel classifiers for recognition of affordances.