EAGER: Deep Causal Representation Learning for Generalizable Visual Understanding

Rensselaer Polytechnic Institute, Troy NY

Investigators

Abstract

The recent developments in deep learning have led to remarkable progress in computer vison and other fields. Current computer vision systems, however, do not perform well in a new environment different from its training environment. This problem, if left unaddressed, could lead to predictive errors and hence significant consequences for life-critical and high-stakes applications such as autonomous driving, robotics, and manufacturing. To address this challenge, this project introduces a novel deep causal learning framework using latent features. Unlike most features that are observable, latent features are hidden. For example, in an object recognition task, latent factors can be vectors describing the scale and poses of objects extracted from the image’s pixels and labels of every training image. The framework leverages deep learning to learn latent feature representation and causal learning to enforce the feature representation to form the direct causes and effects of the target variable that are invariant across domains. In addition to enable computer vision systems to be safely deployable in open-world environments, this project can significantly advance other fields of artificial intelligence (AI) and machine learning, including transfer learning, domain adaptation, and lifelong learning. The project will integrate research with education and involve undergraduate students in this research by leveraging the existing programs at Rensselaer Polytechnic Institute. This research explores a novel idea of integrating causality into deep neural networks to enhance generalization, fairness, and explanation. The project employs the Causal Markov Blanket (CMB) to capture the underlying causal mechanism in the data. It further demonstrates the optimal properties of the CMB features in predicting the target variable. Finally, it introduces an end-to-end deep learning framework to efficiently learn the CMB features for a target variable without causal sufficiency assumption. The project will evaluate the framework on some computer vision tasks. Despite its significant potential, this project investigates a relatively less explored area of study in computer vision and machine learning. The goal of the project is to generate preliminary results to demonstrate its feasibility and to identify areas of improvements. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.