EAPSI: Reconstructing 3D models from 2D images

Cheong Audrey L, Sugar Land TX

Investigators

Abstract

This award supports research aimed at simplifying the process of obtaining 3-dimensional models through enhanced software and algorithm development, enabling 3D model creation on everyday hardware such as a basic camera phone or digital camera. This will allow small projects to perform 3D quantitative studies without having to locate or invest in a 3D scanner. This project will be conducted in collaboration with Dr. Shang-Hong Lai of National Tsing-Hua University in Taiwan. Dr. Lai, an expert in the field of computer vision, has created 3D face models from a single face image. Similar principles can be applied to reconstruct 3D torso models to enhance breast cancer research. Performing surface analysis on the 3D models will facilitate the quantitative analysis of the breast morphology, help improve plastic surgery techniques to adjust shape and volume, and provide 3D visualization of possible surgical outcomes to patients during their pre-surgical consultation. In the broader scheme, a cell phone app could potentially create a 3D model from captured photos on the scene, which will lead to a greater accessibility to 3D data. An algorithm on reconstructing 3D models from multi-view images will be developed for rendering different objects, which will allow for 3D shape analysis. The objectives are to calibrate the images to determine camera positions, construct a coarse depth map of the target object, and modify the parameters of the modeling algorithm to achieve photo-consistency. Multiple photographs of a small toy object will be taken from various positions and these positions will be recorded for reference. Some challenges are differentiating shadow boundaries from structural edges and determining the directions of the light sources. The graph cut method will give the general outline of the 3D model based on the silhouettes from the images. Other methods utilizing illumination and shading will be used to further refine the 3D model. This NSF EAPSI award supports the research of a U.S. graduate student and is funded in collaboration with the Ministry of Science and Technology of Taiwan.