Mixed Precision Arithmetic for Large Scale Linear Inverse Problems

Emory University, Atlanta GA

Investigators

Abstract

The gaming industry, machine learning (ML), and artificial intelligence (AI) are areas that require substantial computational resources and/or require very fast computations, but do not always require high accuracy in certain computational problems. This has motivated companies to manufacture computer chip hardware, such as graphical processing units (GPUs) that can perform very fast computations using low precision computer arithmetic formats. This can result in a 4-times speedup compared to high precision arithmetic used in typical scientific applications. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy. That is, when possible, use low precision for speed, but mix in high precision computations when needed to maintain accuracy. Although previous work has been done for certain core linear algebra computations, relatively little has been done to exploit and understand the implications of using mixed precision arithmetic for the more challenging class of ill-posed problems. The focus of this work is on developing methods for this frontier, so that efficient solvers can take advantage of modern GPUs with mixed precision computing capabilities. Special considerations, which normally do not arise when solving well-conditioned problems, need to be considered. Applications in machine learning, image restoration and image reconstruction, including breast imaging, will be used as target test problems, but an important aim of this work is to construct a computational platform that can be used to efficiently compute approximate solutions of large scale ill-posed inverse problems in a variety of applications. By developing a flexible and adaptable computational platform, the work produced from this project aims to have a broad scientific impact for applications where it is necessary to compute solutions of large-scale inverse problems with regularization, including astronomy, cosmology, geophysics, machine learning, microscopy, and medical imaging. Students and postdocs will be trained as part of this project. The potential for much faster computations has fueled a growing interest in the last decade to use powerful GPU servers for scientific applications, and in particular to use mixed precision algorithms for problems that require high accuracy; that is, when possible, use low precision for speed, but mix in high precision computations to improve accuracy. Recent previous work to develop mixed precision computational approaches for scientific applications have focused on general, well-conditioned linear systems, including iterative refinement, Cholesky factorization and least squares problems, QR factorization, and GMRES. The aim of this project is to focus on the development of mixed precision computations for the more challenging class of large-scale ill-posed inverse problems. The approach will use a combination of operator approximation, using a low precision truncated singular value decomposition with iterative refinement exploiting mixed precision formats to ensure sufficient accuracy, or as preconditioners in Krylov subspace iterative methods. In addition, mixed precision, possibly with iterative refinement, will be applied within flexible and/or inexact hybrid Krylov subspace methods to reduce storage requirements and computational costs that increase at each iteration. The methods developed in this project can be used as tools to obtain approximate solutions of ill-posed inverse problems, or as solvers in machine learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.