CHS: Small: Shape Processing with Deep Architectures

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

Digital representations of three-dimensional shapes are becoming an integral part of many scientific and engineering fields. And 3D printers are becoming increasingly popular for transforming these digital representations into real objects for industrial or home use as tools, mechanical components, or household items. Virtual environments for simulation, education, and entertainment require large numbers of diverse 3D models to maintain realism and operability. In computer vision, several recent object recognition algorithms are trained on large synthetic datasets originating from 3D shape repositories. All these fields and several others increasingly depend on the availability of digital 3D shapes. However, it is still a challenge to develop tools that allow users to easily produce new 3D shapes, or even to retrieve and process existing ones from online repositories. Current 3D modeling tools often require laborious user interaction via low-level selection and editing commands, while existing search engines for retrieving 3D shapes largely depend on manually entered tags and hand-tuned feature representations which results in unsatisfactory retrieval performance. The PI's goal in this research is to create algorithms based on "deep" architectures that automatically learn from data how to reliably analyze and synthesize 3D shapes that are optimized for 3D shape processing and synthesis performance (so that, for example, when these shapes are 3D printed they retain desirable physical properties such as reduced mechanical stress). Project outcomes will be released as open source code and will have broad impact not only on computer graphics and computer vision but also on industry, computer-aided design and mechanical engineering pipelines, and architectural engineering software for buildings and indoor environments. This research will make three major contributions in terms of intellectual merit: 1) Deep architectures and algorithms for learning 3D shape feature representations optimized for retrieval and processing performance; the algorithms will be trained on massive 2D image datasets as well as 3D model repositories, and the learned representations will be used for accurate shape categorization, segmentation, correspondence, style analysis, and texturing. 2) Deep architectures and algorithms for learning to reliably generate new 3D shapes based on intuitive user input, such as sketches and textual descriptions. 3) Algorithms for learning to optimize the underlying geometry of 3D shapes such that they acquire desired physical properties for 3D printing and manufacturing.