NRI: Collaborative Research: Task Dependent Semantic Modeling for Robot Perception

University Of North Carolina At Chapel Hill, Chapel Hill NC

Investigators

Abstract

The research in this project enables robots to better deal with the complex cluttered environments around us, ranging from open scenes to cluttered table-top settings and to perform the basic mapping, navigation, object search so as to enable fetch and delivery tasks most commonly required in service co-robotics applications. The key contribution of the project is to develop visual perception systems for robots that can understand the semantic labels of the visual world at multiple levels of specificity as required by particular robot tasks or human-robot interaction. In addition, the project enables robot perception systems to better understand new, previously unseen, environments through automatically adapting existing learned models, and by actively choosing how to best explore and recognize novel visual spaces and objects. The datasets and benchmarks, as well as the developed models, form basis for more rapid progress on semantic visual perception for robotics. The development of methodologies for learning compositional representations which enable active learning and efficient inference is a long standing problem in computer vision and robot perception. Guided by the constraints of indoors and outdoors environments, we plan to exploit large amounts of data, strong geometric and semantic priors and develop novel representations of objects and scenes. The developed representations are captured by compositional structured probabilistic models including deep convolutional networks. Doing this rapidly is required to support active visual exploration to improve semantic parsing of a space. Furthermore the project team collects and disseminates a large dataset of densely sampled RGBD imagery to support offline evaluation and benchmarking of active vision for semantic parsing. The project can result in advances in active hierarchical semantic vision for robot tasks including exploration, search, manipulation, programming by example, and generally for human-robot interaction.