NRI: Large-Scale Collaborative Semantic Mapping using 3D Structure from Motion

Georgia Tech Applied Research Corporation, Atlanta GA

Investigators

Abstract

The project develops techniques to advance the state of the art in tackling the challenges associated with creating such representations using robots, namely issues related to the scalability and semantic interpretability of such maps. The research activities include advancement of knowledge in multiple fields, such as computer vision, structure from motion, robotics, and semantic mapping. The results have the potential for many societal applications including city planning, asset management, creation of historical records, and support for autonomous driving. The demonstration of the developed theoretical techniques for real-time interaction between humans and robots facilitated by a semantic map enables even greater societal benefit, for example for emergency management, crime prevention, and traffic management. Direct educational impact is anticipated for graduate students and the results are disseminated through both publications and software, allowing the community to leverage the results. This research program advances real-time large-scale distributed semantic mapping of outdoor environments. Specifically, the research team is enabling real-time large-scale semantic mapping by using unsupervised object discovery, obviating the need for large sets of annotated videos for each object category which becomes prohibitive when dealing with hundreds of object categories. The research team frames this process within the structure from motion optimization framework, thereby leveraging geometric and multi-view constraints and features to increase reliability of object track association as well as category clustering. In addition to address scalability, the project develops a distributed, multi-robot system, allowing large teams of air and ground vehicles to cooperatively build a map of large geographic areas in reasonable time frames. Furthermore, the project develops techniques to make the maps more semantically-meaningful and hence interpretable by humans. To accomplish this objective, the research team uses automatic techniques to attach semantic labels to objects discovered in an unsupervised manner. Moreover, humans can interact with the system at multiple levels. Human users can refine both the object categories and semantic labels to increase their accuracy, as well as designate dynamic targets of interest and task robots to track them.