Images with Normals: Acquisition, Analysis, and Depiction

Princeton University, Princeton NJ

Investigators

Abstract

Rusinkiewicz Princeton University Abstract: The development of computer-based methods for stylized depiction of 3D models promises to help scientists and engineers produce clear and compelling illustrations and visualizations. However, despite steady progress on making 3D acquisition inexpensive and practical, obtaining complete 3D models of complex objects remains challenging. This project investigates the creation of illustrations from a data type lying between simple 2D images and full 3D models: images with a surface normal stored at each pixel. These ``RGBN images'' have the potential of becoming a widely-used data type because of the ease, flexibility, and quality with which they may be acquired, and because they contain enough information to permit many analysis and depiction tasks. That is, they combine an acquisition process only mildly more complex than that for digital photographs with the power and flexibility of tools originally developed for full 3D models. Methods for RGBN shape analysis and nonphotorealistic rendering will allow for exploration and communication of surface shape and detail in domains such as medical and technical illustration, art history, and forensic analysis. This project encompasses a comprehensive investigation of the RGBN image data type, with the aim of developing a practical pipeline for acquiring images with normals and generating stylized depictions. On the acquisition side, the project is developing hardware/software acquisition systems for robustly acquiring RGBN images in contexts ranging from millimeter-scale objects through cityscapes, and including both static and moving objects. Next, the project includes a mathematical analysis of methods for signal processing on RGBN images, including scale-space analysis and derivative estimation. These signal processing techniques are used to develop methods for depicting shape and color, including shading, line drawing with suggestive contours and crease lines, exaggerated shading, and enhancement of depth discontinuities. Finally, RGBN analysis and processing algorithms such as texture analysis/synthesis, inpainting, and similarity-based search are being developed.