Collaborative Research: Theory and Applications of Structure-Conforming Deep Operator Learning

University Of California-Irvine, Irvine CA

Investigators

Abstract

The first-principle-based approach has achieved considerable success in numerous engineering and scientific disciplines, including fluid and solid mechanics, electromagnetism, and more. Among its most significant applications are partial differential equations (PDEs) which, in conjunction with their analysis and numerical algorithms, represent some of the most powerful tools humanity has ever developed for understanding the material world. However, increasingly complex mathematical models arising from physics, biology, and chemistry challenge the efficacy of first-principle-based approaches for solving practical problems, such as those in fluid turbulence, molecular dynamics, and large-scale inverse problems. A major obstacle for numerical algorithms is the so-called curse of dimensionality. Fueled by advances in Graphics Processing Unit and Tensor Processing Unit general-purpose computing, deep neural networks (DNNs) and deep learning approaches excel in combating the curse of dimensionality and demonstrate immense potential for solving complex problems in science and engineering. This project aims to investigate how mathematical structures within a problem can inform the design and analysis of innovative DNNs, particularly in the context of inverse problems where unknown parameters are inferred from measurements, such as electrical impedance tomography. Additionally, the programming component in this project will focus on training the next generation of computational mathematicians. The Operator Learning (OpL) framework in deep learning provides a unique perspective for tackling challenging and potentially ill-posed PDE-based problems. This project will explore the potential of OpL to mitigate the ill-posedness of many inverse problems, as its powerful approximation capability combined with offline training and online prediction properties lead to high-quality, rapid reconstructions. The project seeks to bridge OpL and classical methodologies by integrating mathematical structures from classical problem-solving approaches into DNN architectures. In particular, the project will shed light on the mathematical properties of the attention mechanism, the backbone of state-of-the-art DNN Transformers, such as those in GPT and AlphaFold 2. Furthermore, the project will examine the flexibility of attention neural architectures, enabling the fusion of attention mechanisms with important methodologies in applied mathematics, such as Galerkin projection or Fredholm integral equations, in accordance with the a priori mathematical structure of a problem. This project will also delve into the mathematical foundations of attention through the lens of spectral theory in Hilbert spaces, seeking to understand how the emblematic query-key-value architecture contributes to the rich representational power and diverse approximation capabilities of Transformers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.