RI: Small: 3D Nonrigid Object Reconstruction from Large-Scale Unorganized 2D Images

University South Carolina Research Foundation, Columbia SC

Investigators

Abstract

Reconstructing the 3D shape of an object from multiple 2D images is a fundamental problem in computer vision. Prior work on this problem usually requires the object of interest to be rigid or the available 2D images to be well organized, such as consecutive frames in a video. This project investigates the challenging problem of reconstructing a nonrigid 3D object from a large number of unorganized 2D images, which may be taken at different times, with different backgrounds, from different perspectives, under different lighting conditions, and/or using different cameras. The research team develops new algorithms of combining object localization, feature matching, and partial shape matching across the images to segment the 2D object of interest from the input images. The segmented 2D objects are organized into clusters to recover the underlying 3D nonrigid deformation. Pieces of the 3D object are reconstructed from these clusters and finally assembled to obtain the complete 3D object by removing the in-between nonrigid deformations. An image database with 2D images of selected nonrigid objects is constructed for performance evaluation. This research benefits many applications in computer vision, computer graphics, computer gaming, zoology, microbiology, marine science, and medical research, which all involve the modeling of 3D norigid objects. Progress made on object localization, feature matching and partial shape matching has immediate applications in object detection, object recognition, image search, surveillance, tracking, and segmentation. This research also provides an excellent setting for the training of both undergraduate and graduate students.