CHS: Small: Predictive Material Appearance Modeling at Multiple Scales

University Of California-Irvine, Irvine CA

Investigators

Abstract

Recent advances in immersive display technologies, such as virtual reality (VR) headsets, have allowed computer-simulated virtual worlds to be experienced from first-person perspectives. But to offer truly immersive virtual experiences, predictive simulation of richly diverse material appearance is crucial. Specifically, because a material's appearance varies greatly across different physical scales, new methods to model it accurately and consistently at greatly varying scales are needed. The goal of this research is to develop new techniques that will not only enable high-fidelity material appearance in interactive/VR applications (for example, providing the user with an accurate impression of the terrain and the sky in flight simulation, or of various decorative materials such as wood or metal when exploring virtual rooms), but will also benefit offline predictive rendering tasks. To maximize industrial impact, the PI will release the entire software architecture as open source so that it is available to designers, retailers, developers, educators, artists, and students. The project's broad impact will be further enhanced by presenting the findings in workshops and high-profile conference tutorials such as SIGGRAPH/Eurographics courses, and by leveraging the new appearance modeling techniques to develop pedagogical tools (e.g., using VR) for outreach to high-school students (especially minorities) to foster interest in STEM. In computer graphics and vision, many models have been developed to computationally describe and reproduce the appearance of real-world objects. However, these models are generally designed specifically to work at a single, fixed physical scale. Many reflectance models, for example, treat objects as opaque smooth surfaces; these methods work well when viewed from a distance, but suffer from a lack of fine-grained details and irregularities when viewed close-up. Micro-appearance models, in contrast, explicitly capture a material's small-scale structures via high-resolution volumes or meshes. Due to their high complexity, these models are best suited for producing zoomed views of small objects and can be prohibitively expensive to represent large scenes. The inability to work across multiple scales has become a major obstacle to building highly immersive virtual realities. The objective of this research is to develop new computational tools to model and reproduce material appearance efficiently in a consistent and predictive manner across greatly varying scales. To this end, new appearance modeling techniques will be developed for both data-driven and discrete stochastic models, along with scale-bridging algorithms to ensure that the models work efficiently and consistently across multiple scales. To achieve this goal, the following computational challenges will need to be overcome: Nonlinearity - the relation between material models and final appearance is known to be highly nonlinear and difficult to filter and interpolate; Non-locality - complex light transport phenomena such as inter-reflection cause local material changes of model parameters to affect material appearance globally; and High Computational Cost - during scale-bridging it is usually necessary to search for model parameters at different scales while preserving the final appearance, which generally requires solving expensive numerical optimizations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.