Collaborative Research: HCC: Medium: Neural Materials for Realistic Computer Graphics

University Of California-San Diego, La Jolla CA

Investigators

Abstract

Realism has always been an important goal of computer graphics, not only because it makes images more convincing and virtual environments more immersive, but because accuracy is important when renderings are used to make real-world decisions. Realistic images are made by using models to simulate the physical process of light reflection from surfaces in the scene, and state-of-the-art reflection models, while accurate for smooth, homogeneous materials, are quite inaccurate for materials with detailed surface structure. This project explores an entirely new way of modeling surface reflection, by learning the characteristic reflectance patterns of particular classes of materials from many detailed measurements of surfaces. These new models will have broad impact by completely transforming the way materials are modeled for computer graphics in applications like moviemaking, industrial design, marketing, advertising, architecture, virtual reality and home remodeling. This research centers around building models that are more appropriate for describing fine-scale detail. Current “physically based materials” are essentially parametric reflectance models with parameters modulated by texture maps. These can match aggregate large-scale behavior, but not the complexities of real reflectance at close scale. This project explores a fundamentally new way to represent materials, based on neural function approximators. The work will go beyond simply attaching reflectance to opaque objects, by developing thin neural reflectance fields that accommodate fuzziness, translucency and unmodeled geometric detail. The new neural materials will be flexible and general for the full range of materials needed from virtual environments to visual effects, supporting applications ranging from automatic look development from only a few reference photographs, to inverse object modeling from images, combining geometric and material detail. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.