CAREER: Perceptually Guided Hand Motion Synthesis

Clemson University, Clemson SC

Investigators

Abstract

This research will explore ways to automatically synthesize hand and finger animation for virtual characters that exploit human perception as an inherent part of the algorithm. In recent years, character animation has taken tremendous strides towards realistic virtual agents, with increasingly better solutions for body motion capture, for achieving highly realistic facial animation, and for simulating cloth and hair. With these key components in place, the need to create plausible hand and finger motions has become important because these play a crucial role in communicating information while also allowing us to conduct basic tasks and to handle complex tools. But the differences in size and complexity of hand motions compared to body motions make it difficult to capture or synthesize both at the same time. Therefore, finger motions are typically still animated manually, which is a cumbersome process. Taking advantage of perceptual findings could enable the creation of new algorithms to accomplish this task (e.g., by suggesting new methods and by aiding in algorithm parameter adjustment). The ultimate goal of this research is to merge character animation and motion perception into an interdisciplinary field that yields new insights and approaches to finger and hand movement synthesis as well as a better understanding of how we communicate. If successful, the work will significantly advance the way we design algorithms to bring virtual characters to life, and project outcomes will have broad impact not only in computer graphics but also in applications such as virtual reality, robotics and prosthetics. The project includes integrated educational and outreach activities for K-12, undergraduate, and graduate students. This research will initiate a fundamental transformation of how we design algorithms for character animation by coupling perceptual experiments with computer animation algorithm development for hand and finger motion synthesis. Several approaches and devices have been suggested for hand and finger animation, each with their own drawbacks. Some of these approaches show promise and could be improved or combined, but the options are many and a more systematic approach is required. This work will focus on two applications: data-driven hand motion synthesis for virtual characters, and hand motions for interaction and communication in virtual reality. The plan is to develop an algorithm for hand motion synthesis based on segment and pose matching, on perceptual insights on the relevance of finger poses and dynamics, and on the perception of collisions, to support real-time interaction in virtual reality.