CAREER: Visual 3D Acquisition, Modeling and Rendering of the Real World

University Of North Carolina At Chapel Hill, Chapel Hill NC

Investigators

Abstract

The long term goal of this research problem is to tackle these problems. The proposed project is to develop automatic visual 3D modeling approaches that can deal with objects, people and complete environments, including complex shapes, reflectances and non-rigid motions. The resulting models should make it possible to render new images, possibly with different lighting, and to perform 3D measurements, opening up a wide range of applications. An important aspect will consist of also recovering camera motion and calibration during acquisition. More specifically the research will address the following topics: (1) removing limitations of existing approaches and increase flexibility and speed, (2) extension from static to dynamic scenes using single and multi-camera approaches, (3) extension towards capturing complete (immersive) environments using standard and/or panoramic cameras and (4) extensions towards recovering more complex reflectance properties and disentangling reflectance and illumination. This research will be developed towards making it possible to simply capture the appearance of real objects, people and environments, adapt some properties if necessary (e.g. pose, lighting) and then seamlessly (re)combine different real and virtual scene elements. The educational component forms an integral part of this project. Existing courses will be further enhanced and additional courses will be developed to provide a complete computer vision course offering. Important aspect will be to ensure integration and complementarity with the computer graphics courses provided within the department and to closely link courses and research. To ensure a broader dissemination, I intend to publish a book and to organize courses at specialized conferences as well as to provide application results for exhibits open to the broader public.