Computational Methods for Hierarchical Manifold Learning

George Mason University, Fairfax VA

Investigators

Abstract

The enormous practical success of deep learning warrants a clear and concise mathematical explanation. Although the components of a neural network, and the rules for propagation of information through the layers, are extremely simple, there is to date a lack of a corresponding deep understanding of the roles of the various mechanisms involved. Second, there is a lack of transparency: While a given set of network weights may fit scientific or engineering data in-sample and even generalize well out-of-sample, using a neural net to explain the data or the system producing the data is usually very difficult or impossible. In this project, the PI will leverage recent progress in mathematical methods from applied and computational harmonic analysis to develop a hierarchical algorithm, based on manifold learning, to replicate the strikingly successful properties of deep learning while adding improved accuracy, adaptability to data, smoothness priors, and transparency. A sequence of computational projects is planned to develop a hierarchical algorithm for deep learning, based on representing manifolds by the Laplace-Beltrami operator. Proposed work supports the construction of a complete algorithm that uses layers of manifold learning kernel methods to represent data in a deep manifold learning infrastructure. The development of the learning algorithm consists of three parts: (1) the construction of a hierarchical manifold learning architecture, using eigenfunctions of the Laplace-Beltrami operator to represent data, and replicating the sharing and pooling features of neural networks, (2) building innovative algorithms for the purpose of optimizing the solution to identification problems, and (3) development of resampling methods to handle large data sets.