CAREER: A Novel Blueprint for Representation Learning of Relational Invariances

Purdue University, West Lafayette IN

Investigators

Abstract

Graphs, tensors, and logical formulae are three of the most fundamental mathematical abstractions used to model complex data dependencies. These constructs are essential for the design of many modern AI innovations such as recommender systems in social networks, robots that reason about their environment, using generative models to design new drugs, or extracting business rules from data. Our theoretical understanding of graphs and tensors has significantly advanced in the past century, and the opportunity now exists for the development of practical tools for day-to-day machine-learning tasks. This research project aims to make available to neural network architectures data representations that are sufficiently expressive for complex tasks and computationally tractable for more tasks. This project leverages the PI’s prior work into a novel blueprint for better modeling complex relational input data, thereby translating the modern theoretical interpretation of sets, graphs, tensors, and logical formulae into provably more capable practical tools. The key insight is to leverage invariant theory to develop a novel framework for representation learning of complex relational data via approximate invariances. The full development of the framework of this proposal holds promise to harness the synergy between invariant theory and computational methods. Specifically, this study investigates: (a) most-expressive tractable models through ergodic theory and variational approximations; (b) adaptive model tractability and expressibility through invariance relaxations and quasi-invariant neural architectures; and (c) novel model evaluation metrics and tests of expressiveness for invariant representations. This project also explores the application of these methods on tasks defined on temporal graphs and tensors, sets and multisets, hypergraphs and simplices, logical formulae, and molecules. This project will also support substantial outreach activities, including workshop organization, course development, and the recruitment of underrepresented minorities to STEM careers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.