CISE Research Resources: The LiveActor Virtual Reality Environment

University Of Pennsylvania, Philadelphia PA

Investigators

Abstract

EIA 0224432 Badler, Norman I. University of Pennsylvania Title: CISE RR: The LiveActor Virtual Reality Environment This proposal, creating, controlling, and interacting with real-time embodied virtual human agents, aims at supporting research projects that require interactivity support to Detect, model, and describe human participant motion either for building action models or for providing real-time input to interpersonal interactions, such as training and Immerse the human participant in a visual environment conducive to the execution of actions in a context of objects and both real and virtual people. Components will be purchased to construct an immersive room for real-time participants in interactive experiences. The room will consist of an Ascension ReaCTor motion capture system and a 4 surface rear-projection room, called the LiveActor. In turn, this facility supports at least three research projects: Computational Models of Verb Semantics. Real Time Decision Critical Training, and Virtual Animated Environments from Language. The first project utilizes a Parameterized Action Representation (PAR) that holds computational definitions of human and other agent motions. PAR is used to both synthesize animations and recognize presence in a motion captured input stream. An immersive environment in which participants interact with each other and with virtual agents provides opportunities for capturing, characterizing, and representing genuine physical and emotional actions, and for employing these actions to affect and control reactive behaviors in virtual agents. The second project utilizes the graphical portrayal of embodied agents to close encounter training requiring the user to analyze and react to facial actions, body posture, and gesture quality. Fundamentally different from VR navigation and exploration, these interactions require realistic human models and detailed, variable, controllable parameters. The last project encourages exploration, but not construction and animation, since the necessity to design and script VR worlds in advance constitutes a bottleneck. This project proposes an "imagination" machine in which users create, populate, and animate their own virtual worlds. Natural language that understands descriptions of situations and action, depicts the graphical arrangements, and sets current and future context-dependent behaviors into its animated agents, serves as the primary ingredient in this system. The immersive LiveActor space allows a user to input her own body motions for specific yet parameterizable movements for characters and their interactions.