Collaborative Research: Sparse Optimization for Machine Learning and Image/Signal Processing

Syracuse University, Syracuse NY

Investigators

Abstract

Data sets involved in information technology, nanotechnology, biotechnology, civil infrastructure, environmental science, and other important areas are often extremely large. Due to growing quantities of data and related model sizes, demands for more competent data processing models continue to increase. Data sets in applications often have certain embedded sparsity structures, in the sense that their essential intrinsic characteristics can be represented by smaller amounts of information. Motivated by this observation, the aim of this project is to develop computationally efficient methods for non-smooth and non-convex optimization by exploring sparsity structures embedded in data sets. The investigators anticipate that the outcomes of this project will be of use in many application areas. The research and associated educational components in this project are expected to provide undergraduate and graduate students rigorous training so that they will have the skill sets needed to face the scientific and technological challenges of the big data era. This project will address several critical issues in non-smooth, non-convex optimization that result from sparse modeling of data in a range of applications, including machine learning and sparse image/signal restoration. For both learning and image/signal restoration, proper sparse regularization models will be developed. Suitable sparsity promoting functions will be designed for use in forming regularization terms, and appropriate bases/transforms will be constructed so that the resulting regularization algorithms compel their solutions to be sparse. For machine learning, appropriate reproducing kernel Banach spaces will be built up to allow for the representation of complex and rich geometric and topological structures of data and hence lead to improved learning outcomes. Representer theorems of the resulting learning methods in these spaces will be established so that the learning solutions can be expressed as a combination of a finite number of kernel sessions with the number equal to that of the data points used in training even though the hypothesis space is of infinite dimension. Geometric features of these spaces will be employed to induce sparsity of the learning solutions. The construction of bases or transforms via machine learning from data is expected to lead to improved methods for image/signal restoration. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.