CAREER: Sensory Information for Interaction in Single-User and Collaborative Virtual Environments

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

When virtual environments achieve their full potential, they will have the capacity to make a significant impact in several promising application domains including scientific visualization, medical diagnosis and surgical training, flight simulation, educational tools, and 3D games. However, due to technological limitations, the graphic and haptic feedback available in current interactive virtual environments cannot match the sensory feedback available in the natural environment. In virtual environments, providing an abundance of integrated sensory feedback to users comes at a significant cost. Technological limitations, such as system latency, cumbersome tracking/feedback equipment, and computationally expensive graphics and haptic rendering, make it impractical to provide subjects with rich sensory feedback about all aspects of interaction. Further, these limitations are amplified when we consider a collaborative system where multiple users share the environment across a network. The PI's premise is that virtual environments will only reach their full potential when we have a deeper understanding of the human user. Because humans constitute an essential component in the interaction equation, knowledge about human behavior is essential to the design and evaluation of these systems. In this project, the PI's goal is to understand how sensory feedback conditions in virtual environments affect the planning and performance of simple and collaborative object manipulation movements. A multi-disciplinary approach, using methodologies from computer science, engineering, human motor control, biomechanics and neuroscience, will be used to achieve this purpose. Specifically, the PI will investigate what sensory (graphics and haptic) information users require to move effectively in single-user and collaborative computer-generated environments, when that information is required, and how best to present the information so that it can be effectively used. Basic motor control measurement and predictive modeling techniques will be employed to understand how sensory feedback is used, to quantify the negative effects of lag, and to suggest methods for improving the presentation of sensory information to users under a variety of task-specific conditions. By gaining a comprehensive understanding of the role of graphics and haptic information for interaction in computer-generated environments, the PI expects it will be possible to make evidence-based recommendations to designers of virtual environments regarding how to provide the most effective sensory feedback to users at the most efficient times. Broader Impacts: Beyond the research activities, this project includes several educational goals. Because the PI believes that all students should be exposed to research efforts being undertaken at their university, she will ensure that students in her undergraduate and graduate classes spend at least one session collecting data, participating in an experiment, or observing the research process. . New and innovative approaches, including active learning techniques and web-based teaching, will be investigated to make the teaching/learning process active and interesting.