EAGER: Creating an Unsupervised Interpretable Representation of the World Through Concept Disentanglement

Duke University, Durham NC

Investigators

Abstract

Humans are able to break down a large entity into smaller and simpler concepts, just from having seen many objects and their relationships. Reproducing this type of behavior in a machine learning model has several benefits. In particular, it could lead to computational ways of representing the world that are interpretable yet powerful. These new representations could be used within machine learning algorithms, allowing the algorithms to be more robust and more likely to generalize when the underlying situations change. For instance, if an algorithm has found a collection of parts that an object is typically comprised of, then it can use those parts to identify this type of object even when it is in an unusual setting, or when the object itself is unusual. This new way of representing the world will allow more robust and generalizable machine learning models. This will be particularly helpful for difficult challenges in computer vision, including problems related to vision systems in automated vehicles, analysis of medical time-series, and materials science problems related to the understanding of material properties and discovery of new materials. Specifically, the main goal of this project is learning with interpretable learned concepts using a disentangled neural network. The approach breaks the problem down into three steps that each could be manageable, and each step can be checked and improved independently of the other steps. The steps are to decompose each observation into local parts, identify possible concepts by looking at common relationships between the local parts, and align the proposed concepts, based on their semantic meaning, within a disentangled neural network. The discovered concepts will be interpretable and can be used as features for many downstream tasks. The disentangled neural networks built from these concepts could potentially generalize more easily to new situations than other approaches. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.