CAREER: Towards Polarimetric Visual Understanding

George Mason University, Fairfax VA

Investigators

Abstract

Polarized light is very common in our surroundings. As most visual inference algorithms leverage the color and brightness of images, the use of polarization in computer vision has not yet reached its full potential. This CAREER project will study how the way surfaces reflect polarized light can be used to help recognize objects more effectively. The project will address the following two questions: How does the polarization of light change after interacting with various types of surfaces? What can polarized light tell us about the kinds of objects that it has interacted with? Addressing these questions can lead to significant improvements in machine vision systems by strengthening their capability to do geometric and semantic scene understanding in which shapes are described and named. This project will result in new polarization-based vision systems for autonomous navigation and smart manufacturing. It will also produce novel solutions to the challenging problems of imaging and sensing through scattering media, including water, fog, haze, clouds, and bodily tissues. This project will investigate polarimetric light transport and tackle the problem of polarimetric visual understanding through analysis by synthesis. The research team will derive theoretical models for polarimetric surface reflectance and volumetric light transport, design visual inference algorithms for scene understanding, and develop computational imaging systems for real-world data acquisition. Specifically, the project will first study the transfer of polarization state upon local surface interactions. Physics-based layered material models will be adopted for characterizing complex surface scattering effects beyond direct mirror reflection. The project will then study the transport of polarized light through a spatial volume. A bottom-up approach will be taken to develop a voxel-based light transport model to mitigate the ill-posed transport matrix decomposition problem. The project will also study the challenging scenarios of turbid media, which could result in new techniques for imaging through scattering media. Leveraging the polarimetric light transport models, visual inference algorithms will be designed and developed for understanding scene properties, such as depth, shape, reflectance, and semantic composition. This project will develop novel computational imaging systems for acquiring real-world data to validate and analyze our proposed models and algorithms. A high precision polarimetric reflectance of various surfaces will be collected in this project and shared with the research community. Other broader impacts of this project include integrating the research results into existing and new course curricula on computer vision. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.