Elements: Software: NSCI: A high performance suite of SVD related solvers for machine learning

College Of William And Mary, Williamsburg VA

Investigators

Abstract

The accrual of vast amounts of data is one of the defining characteristics of our century. With the help of computers, scientists use this data to make and test hypotheses, draw inferences, predict complex phenomena, and make educated policy decisions. Machine learning (ML) is an area in computer science that uses statistical methods to allow computers to "learn" from data, with and without human supervision. Central to the application of machine learning methods is the numerical computation of the Singular Value Decomposition (SVD) of matrices of very large dimension, often larger than a million or even a billion. Since "off-the-shelf" algorithms and SVD software, however, cannot handle matrices of very large dimension, iterative methods used in scientific computing are more appropriate. Yet their stringent approximation quality requirements are often excessive for downstream applications, and result in slow execution times. Recently, methods based on randomization have improved execution times, but their implementations relax the approximation quality, often to detrimental levels. This project proposes to develop a software package that unifies randomized and iterative methods with a particular focus on the specific requirements of various ML applications and with high performance optimizations for modern computing platforms. This will allow scientists to analyze significantly larger datasets, ML researchers to study large models that could not be tackled before, and ML service providers to use the new solvers to reduce their operational cost. This project proposes to develop a software package that unifies randomized and iterative methods with a particular focus on the specific requirements of various ML applications and with high performance optimizations for modern computing platforms. This will allow scientists to analyze significantly larger datasets, ML researchers to study large models that could not be tackled before, and ML service providers to use the new solvers to reduce their operational cost. Specifically, the software package builds upon the state-of-the-art eigenvalue/singular value software package PRIMME that integrates cutting-edge iterative methods and high-performance implementations. The development of the package consists of two thrusts: (T1) Unifying state-of-the-art algorithmic techniques including randomized, streaming, and iterative methods, to deliver consistent experience for a diverse range of matrices with different quality requirements, hardware platforms and precisions, and programming environments. (T2) Developing software devices that enable downstream systems and SVD solvers to interoperate so that users can tune and customize solvers without being experts in numeric linear algebra. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.