HCC:Small: Neural Shape Generators under Geometric, Physical, and Topological Priors

University Of Texas At Austin, Austin TX

Investigators

Abstract

Shape synthesis—creating formal descriptions of novel 3-D shapes—is a foundational area of computer graphics. With the advent of deep learning and generative AI models, computer graphics innovators have adopted machine learning (ML) technology in service of shape synthesis. Current approaches focus on adapting image- and video-generation techniques developed by the computer vision and ML communities. These approaches often produce visually appealing results, but suffer fundamental limitations; they often fail to capture geometry and topology properly leading to unnaturally distorted shapes, or shapes that incorrectly incorporate holes or disconnected pieces. Similarly, existing methods offer no guarantees that shapes synthesized will be physically suited for manufacturing. These limitations place fundamental barriers to applying machine learning methods for shape synthesis in applications such as augmented and virtual reality, embodied AI (such as robotics), and manufacturing (including 3-D printing). This project addresses the deficits in current methods by developing a computational framework that incorporates physical, topological, and geometrical preferences when learning shape synthesis. The overarching goal of this project is to establish shape synthesis as a scientific sub-community that departs from simple applications of 2-D image-generation techniques. Integrated education and outreach activities amplify the broader impacts of this project. The key idea of our framework is to model various geometric, physical, and topological priors as regularization losses in learning shape generators to enhance their generalizability. We focus on the latent diffusion paradigm that has led to state-of-the-art shape generators. The proposed research consists of two thrusts. The first thrust studies principled approaches that enforce geometric, physical, and topological priors to improve the diffusion procedure. The second thrust focuses on improving the shape decoder by modeling regularization losses that enforce these priors. We seek to revolutionize 3D shape generation from the current focus of visual appearance to synthetic shapes that are geometrically feasible, physically stable, and topologically correct. Toward this goal, the project will develop differentiable tools in structural shape analysis, computational topology, and shape analysis that can be easily integrated into learning shape synthesis models. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.