CAREER: Leveraging the Virtualness of Virtual Reality for More-Effective Training

University Of Texas At Dallas, Richardson TX

Investigators

Abstract

This research takes a new approach to computer-based virtual reality (VR) training by leveraging the virtualness of VR to provide training solutions not possible in the real world. The aim is to develop VR techniques that are more effective and more time-efficient for training than real-world exercises and VR systems that emulate them. To demonstrate the general effectiveness of these new VR training techniques, the research focuses on two training domains: (1) Robotic operating room tasks for non-surgeon team members, including preoperative setup, intraoperative tasks, and postoperative cleanup. (2) Pre-shift inspections of off-highway trucks, which include walk-around inspections, mounting the vehicle, and in-cab inspections. Formative expert reviews, studies comparing the new techniques to traditional VR training versions, and real-world validation studies will be used to determine the effectiveness and efficiency of the techniques. Educational activities include a graduate VR course, research experiences for undergraduates, a summer research program for underrepresented high school students, an annual VR summer camp for high and middle school students, and the creation of online video tutorials for VR development to support formal and informal learning. A new framework for designing and analyzing VR training techniques has been developed to facilitate this research, the Framework for Altering Fidelity to Influence Memory. It connects two concepts, system fidelity and working memory. System fidelity is the objective degree of exactness with which real world experiences are reproduced by a VR system. Working memory is the temporal activation of short-term or long-term perceptual, cognitive, and motor memory networks in the mind of the human user. The premise of the framework is to manipulate an aspect of system fidelity to help a stage of working memory, which in turn should improve the effectiveness of training. Six novel VR techniques, based on approaches identified with this framework, will be developed and investigated: (1) causation signaling, (2) sensory accents, (3) situational dramatization, (4) error instigation, (5) object conjuration, and (6) regulated biomechanics. For example, the causation signaling method is to purposefully alter time and physics to convey the cause-and-effect relationship between an event and its consequence. Time warping is a causation signaling technique in which the training scenario can be fast-forwarded after an error to the resulting consequence and then rewound back to the decision point just before the error. This technique will convey to the user the cause-and-effect relationship between the error and resulting consequence, in addition to allowing the user to correct the mistake.