Collaborative Research: Algebraic Framework of Compositional Functions for New Structure, Training, and Explainability of Deep Learning

University Of California-Santa Cruz, Santa Cruz CA

Investigators

Abstract

Deep learning is a method of machine learning inspired by the human brain. Data is fed to a multi-layered (deep) network of trainable 'neurons' and the network is then trained to model complex relations and processes. Deep learning has had impressive success in applications such as image recognition and natural language processing. And yet, there are few theoretical guarantees to the method, to provide assurances in regards to performance features such as error and to explain its success. This is an impediment to broader application of deep learning, as many potential applications require guarantees for safety, reliability, and accuracy. This project pursues a solid mathematical foundation for a better understanding of the explainability of deep learning, to enable more efficient neural network design and training algorithms that benefit a wide range of applications. The project also includes a significant educational component that is designed to foster interdisciplinary education by engaging undergraduate and graduate students from the investigators' departments (Applied Mathematics, Electrical Engineering, Computer Science, Mathematics) in the proposed multidisciplinary research. The project includes plans to promote diversity, equity and inclusion in STEM education at the University of California Santa Cruz and the University of Texas at San Antonio, which are both Hispanic Serving Institutions. The overarching goal of this project is to develop a unified algebraic framework and approximation theory for deep neural networks so that the framework is applicable to a wide spectrum of problems including regression, solving differential equations, designing optimal feedback control, and computer vision. The proposed research is motivated by the fact that most complicated and high dimensional input-output relations in real-world applications can be represented as compositions of simple low-dimensional functions. Thus, compositional functions, including deep neural networks, serve as a natural way to describe complex high dimensional functions. Representing compositional functions as layered acyclic graphs, the project will explore the compositional features of the problems to be solved by machine learning; study the error propagation in layered acyclic graphs; and investigate the interconnection between the compositional features and the fundamental issues of machine learning, such as the error bounds in universal approximation, deep neural network design and training, and validation and explainability. The algebraic framework, approximation theory, and computational algorithms to be developed in this research project should advance the design, training, and mathematical foundations of deep learning. They seek also to be directly applicable to a wide spectrum of applications including feedback control design and computer vision, which are included in this project, as well as other important machine learning applications, such as regression and data-driven modeling of dynamical systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.