ITR/SY: Electronic Books for the Tele-immersion Age: A New Paradigm for Teaching Surgical Procedures

Brown University, Providence RI

Investigators

Abstract

Tele-immersion will provide a dramatic new medium for groups of people remote from each other to work and share experiences together in an immersive 3D virtual environment, much as if they were co-located in a shared physical space. Immersive electronic books that in effect blend a "time machine" with 3D hypermedia, will add an additional important dimension, that of being able to record experiences in which a viewer, immersed in the 3D reconstruction, can literally walk through the scene or move backward and forward in time. While there are many potential application areas for such novel technologies (e.g., design and virtual prototyping, maintenance and repair, paleontological and archaeological reconstruction), the focus here will be on a societally important and technologically challenging driving application, teaching surgical management of difficult, potentially lethal, injuries. Today, the pace of surgical innovations has increased dramatically, yet the mechanisms for training and re-training suffer from inflexible timing, extended time commitments, and limited content. Traditional videotaped instruction has long been available to help surgeons learn new procedures, but this approach is only marginally effective due to the fixed point of view that is integral to the narration, lack of depth perception and interactivity, and missing information; in short, the experience of watching a video is not sufficiently close to being there and seeing the procedure. In this project the PI will develop a new paradigm for teaching surgical procedures that allows surgeons to witness and explore (in time and space) a past surgical procedure as if they were there, with the added benefit of instruction from the original surgeon or another instructor, as well as integrated 3D illustrations, annotations, and relevant medical metadata. The trainees should be able to freely and naturally walk around a life-sized, high-fidelity, 3D graphical reconstruction of the original time-varying events, pausing or stepping forward and backward in time to satisfy curiosity or allay confusion. To make this reality, the PI and his team bring together experts in several disciplines, and will be able to collectively leverage their prior work in tele-immersion, time-varying 3D scene capture, interaction metaphors, "cinematic" techniques. and authoring tools.