RI: Small: Crowd-Sourcing the World: Scalable Methods for Dynamic Structure from Motion

University Of California-Santa Barbara, Santa Barbara CA

Investigators

Abstract

This project enables global-scale dynamic reconstructions that can scale to eventually encompass all of the world's 3D data, to which any user may contribute new visual data, thereby ensuring a more complete, up-to-date model and a better experience for users of applications that rely on such data. The abundance of publicly available imagery from a variety of sources (consumer, industry, and government) and the proliferation of networked mobile devices equipped with cameras provide an opportunity to build large-scale 3D models that cover the entire world. Such global 3D models are invaluable to a wide range of applications that require real-time access to environment structure, such as providing assistance to the visually impaired, exploring dangerous areas for search-and rescue operations, urban planning, self-driving cars, and virtual tourism. The key research question driving this work is how to efficiently and accurately update a global-scale 3D reconstruction in order to support a dynamic global model. The research uses structure from motion (SfM) techniques to acquire global context for 3D modeling and then propagate local 3D reconstructions and other visual data back to the global model via a new technique called "globalization." The project addresses a major gap in large-scale SfM: rapidly extending and updating reconstructions to promote real-time use. The project is built on preliminary work in large-scale image matching, localization, and 3D reconstruction. The research team carries out extensive data collection and experimentation to benchmark the performance of the developed techniques and to assess the progress of the project and the utility of the methods. The project is developing a prototype mobile system for contributing imagery and browsing and updating models for public use once it is available.