3d Perception of Specular Surfaces

California Institute Of Technology, Pasadena CA

Investigators

Abstract

The recovery of the shape of reflective surfaces from images is investigated. Both the computational foundations as well as human visual perception are explored. Key issues on the computational front are : (i) the geometrical relationship between surface shape and observations of a reflected scene on the surface; (ii) the nature and role of constraints that help in reconstructing shape from visual measurements. This second issue is investigated first under stringent assumptions (calibrated known scene) which are then lightened progressively (un-calibrated known scene, un-calibrated unknown scene). The relevance of additional visual measurements (multiple images from stereoscopic rigs, occluding boundaries, internal boundaries) as well as the relevance and use of statistical constraints (generic viewpoint assumption, isotropy, homogeneity) is also explored. Human perception of mirror surfaces is little explored and very poorly understood. The first area of investigation is qualitative shape perception in the presence of an increasing number of cues (image patch, surface boundaries, reflected scene, internal boundaries) and with different scene statistics both natural and synthetic (regular periodic patterns, isotropic textures, indoor scenes, outdoor scenes). The cues that lead the human visual system to classify surfaces as specular vs. textured/matte are explored next. A third issue is the relationship between the mechanisms underlying shape-from-texture and shape-from-specularities. The proposed research provides: i) methods to measure the shape of specular surfaces, a notoriously hard problem in computer vision; ii) fundamental understanding of the geometry and statistics underlying vision of reflective surfaces; iii) exploration of the value of prior knowledge in a Bayesian framework; iv) insight into an underexplored ability of the human visual system. The broader impacts of this proposal include extending the applicability of 3D scanning system to specular surfaces, which are common in engineering, medicine and art conservation. For this reason methods that are general, practical and low-cost are of particular interest in this study.