EAGER: Leveraging Synthetic Data for Visual Reasoning and Representation Learning with Minimal Human Supervision

University Of California-Davis, Davis CA

Investigators

Abstract

This project investigates how synthetic data created using computer graphics can be used for developing algorithms that understand visual data. Synthetic data provides flexibility that is difficult to obtain with real-world imagery, and enables opportunities to explore problems that would be difficult to solve with real-world imagery alone. This project develops new algorithms for reasoning about object occlusions, and for self-supervised representation learning, in which useful image features are developed without the aid of human-annotated semantic labels. The project provides new algorithms that have the potential to benefit applications in autonomous systems and security. In addition to scientific impact, the project performs complementary educational and outreach activities that engage students in research and STEM. This research explores novel algorithms that learn from synthetic data for visual reasoning and representation learning. While the use of synthetic data has a long history in computer vision, it has mainly been used to complement natural image data to solve standard tasks. In contrast, this project uses synthetic data to make advances in relatively unexplored problems, in which ground-truth is difficult to obtain given real-world imagery. The project consists of three major thrusts, each of which exploits the fact that a user has full control of everything that happens in a synthetic dataset. In Thrust I, it investigates a novel approach to representation learning using synthetic data, and in Thrust II, it extends the algorithm to disentangle task-specific and general-purpose features. Finally, in Thrust III, it explores a novel approach for reasoning about object occlusions.