RI: Small: Inverse Rendering by Co-Evolutionary Learning

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

This project addresses the problem of inverse rendering: recovering 3D shape, material, and lighting from a single image. Inverse rendering is a fundamental problem in computer vision; it recovers the basic properties of a visual scene, and serves as a foundation for higher-level scene understanding such as recognizing objects, actions, and functionalities. Despite its fundamental importance, inverse rendering remains difficult. Solving inverse rendering can significantly advance computer vision and benefit a wide variety of applications from autonomous driving to assisting the visually impaired. This project develops new machine learning algorithms to advance the state of the art of inverse rendering. In addition, the project contributes to education and diversity by integrating research results into courses at various levels and by recruiting underrepresented groups to participate in this research. This research advances inverse rendering technologies using computer graphics and machine learning. In particular, the research team develops two machine learning systems that co-evolve as adversaries: a rendering system that learns to compose 3D scenes and renders images using a graphics engine, and an inverse rendering system that learns to recover shape, material, and lighting from the rendered images. To develop the rendering system, the research team investigates new learning algorithms for adaptive, automatic scene composition. To develop the inverse rendering system, the research team investigates new learning algorithms that integrate neural networks and physics-based vision.