View Synthesis for Dynamic Scenes, With and Without Reconstruction

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

This research investigates the problem of synthesizing novel views of dynamic, three-dimensional scenes containing multiple rigid objects in motion, given a set of photographs captured at different positions and different times. Image-based interpolation and extrapolation methods are developed so that physically-correct motion sequences can be synthesized showing how the scene objects could have moved during the missing time interval. Motion classes corresponding to basic types of rigid motion an object can undergo (e.g., translational motion or rotation about a single axis) will be used. For each class, techniques for synthesizing physically-correct views without first recovering the scene geometry, will be designed, implemented and tested. As an alternative to non-reconstructive image-based view and motion synthesis, methods that utilize object motion information to perform camera self-calibration and 3D scene reconstruction will also be explored. By recovering properties such as the vanishing point and the fundamental matrix for each moving object, camera calibration information can be directly computed without necessitating point correspondences. Results will include a better theoretical understanding of image-based methods for dynamic scenes, and new algorithms for motion interpolation and multi-view, motion-based scene reconstruction.