CAREER: Advancing Constrained and Non-Convex Learning

Texas A&M Engineering Experiment Station, College Station TX

Investigators

Abstract

Machine learning has emerged to be an indispensable tool for addressing many decision-making problems, e.g., autonomous driving. As applications of machine learning algorithms for decision-making broaden and diversify, the requirements on security, fairness, interpretability and generalization have been pushed to higher standards. These emerging issues have brought great challenges to the design of machine learning algorithms in the presence of big and complex data. Traditional machine learning methods by minimizing an unconstrained or simply constrained convex objective have become increasingly unsatisfactory. This project seeks to advance learning with complex objectives and constraints by designing and analyzing efficient and effective optimization algorithms for addressing computational challenges in new machine learning paradigms. The project will enhance the ability to solve large-scale, real-world problems from more diverse and broad applications. Furthermore, the project will strive to communicate the significance of machine learning and optimization and provide excellent research experience to students at different levels. Although both constrained optimization and non-convex optimization have been studied and applied to machine learning in the literature, great challenges and many problems remain unaddressed. The primary focus of this project is to design and analyze a set of efficient optimization algorithms and statistical learning methods for advancing machine learning with complex objectives and constraints at large scale. The technical aims of the project are divided into two thrusts. The first thrust is to (i) develop faster and provable stochastic algorithms for learning with complicated non-convex objectives, and (ii) improve the generalization performance of deep learning by advanced regularization and compression methods through design of efficient optimization algorithms. The second thrust is to (i) design computationally efficient constrained optimization algorithms for learning with complicated and complex constraints, and (ii) investigate their applications in adversarial learning, fair learning, interpretable learning, etc. The optimization tools and techniques developed will enable more advanced regularization and loss minimization methods in machine learning, and should greatly influence other areas, such as operations research, signal processing, data mining, etc. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.