ITR: Collaborative Research: Image-Based Rendering in Forensic Reconstruction and Historical Preservation

University Of Virginia Main Campus, Charlottesville VA

Investigators

Abstract

Valuable information about the world around us is being lost. Every day we lose information of both current and historical significance, simply because it is too difficult to capture 3D models of our environment. Today, such models are primarily created by hand, making measurements with tape measures and entering the results manually. Even after models are constructed, we do not know the best ways to visualize them effectively. This research proposes to design and develop an end-to-end solution to the problem of documentation, dissemination, and display of real-world environments. To focus the research, two driving problems are targeted: forensic science and education. This research brings together an interdisciplinary team including the law enforcement community, the Federal Bureau of Investigation and the Armed Forces Institute of Pathology. Additionally, one of our team members, from the New Orleans Museum of Art, is directing a major exhibition in which we propose to include a virtual environment of Monticello, President Jefferson's home. We will tackle the problems of acquiring, processing, representing, rendering and display of real- world data, focusing on the two target applications. Our goal is to design and build a complete end-to-end prototype system. The work includes: Acquisition. We will mount an accurate rangefinder on a motorized cart and tackle the problem of capturing as complete a model as possible. This requires accurate tracking and a solution to the next best view problem in a practical sense, including resolution, reflectance, and importance of the surfaces. Processing. A very difficult problem is the registration of color with depth data, and of scans from multiple locations. We will develop algorithms to robustly register the data, and filter to reduce outliers and noise. Representation. We will develop algorithms to process the massive amounts of data (100 MB per scan, tens of scans per room) into both efficient geometric and image-based representations. We will compare and contrast the representations in terms of quality and efficiency. Rendering. We will develop hierarchical and out-of-core algorithms to render the image- based and geometric models at interactive rates, and research perceptually driven rendering techniques to maximize rendering efficiency. Display. This is perhaps the most difficult part of the problem because it involves the way humans perceive the environment. We will develop a hybrid projector/head-mounted-display system to provide stereo views to multiple viewers in a life-sized walking environment. The goal is to give the participants a sense of presence, of being in Mr. Jefferson's house.