Collaborative Research: Differential Equations Motivated Multi-Agent Sequential Deep Learning: Algorithms, Theory, and Validation

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

Sequential data observed from multiple agents is ubiquitous in artificial intelligence (AI) and scientific applications, for example, in computer vision, natural language processing, robotics, computational biology and biophysics, and knowledge graphs. Learning from sequentially observed data often provides a global understanding of the underlying system and yields more reliable predictions than learning from a non-sequentially (single-shot) observed data. Sequential data is often irregularly-sampled in time and space and when this is combined with the interaction between agents, it raises tremendous challenges for machine learning. This project addresses these challenges by developing new mathematical understandings of these bottlenecks combined with new mathematically-principled deep learning algorithms for sequential and graph learning. Anticipated results and algorithms from this project will have broad applicability to important societal issues, such as pandemic spread, cooperative robotics, and environmental change. The project includes research training opportunities for graduate students. This project bridges ordinary differential equations (ODEs) and partial differential equations (PDEs) theory with multi-agent sequential learning practice. The project further leverages ODE and PDE insights to advance theoretically-grounded algorithms for deep sequential and graph learning. This project synergistically integrates recent advances in neural ODE methods with recent advances in graph networks for machine learning. The project develops and explores building next-generation algorithms based on wave equations on graphs, coupling second-order continuous dynamics in time with graph filtering. The research includes theoretical guarantees for the new methods in overcoming the over-smoothing issue, to enable sequential learning on graphs with deep architectures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.