CAREER: Structured Minimax Optimization: Theory, Algorithms, and Applications in Robust Learning

University Of Texas At Austin, Austin TX

Investigators

Abstract

This proposal seeks to extend the mathematical theory of minimax optimization, a very commonly used approach that has played a pivotal role in advancing the fields of information theory, machine learning and signal processing. Notably, there has been a recent surge of interest in minimax optimization due to its critical relevance in artificial intelligence (AI), where it can be used to make deep neural networks more resilient against adversarial disturbances in the underlying distribution of data. While there has been recent progress in enhancing the theory of minimax optimization and its algorithms, a notable gap persists in the applicability of this theory to real-world AI scenarios. Existing algorithms and theory primarily focus on the so-called convex-concave optimization setting, while contemporary learning applications frequently entail minimax problems that do not adhere to this structure and are more complex. This project aims to develop efficient methods for minimax optimization for AI by capitalizing on the unique structure of the AI prediction function. This interdisciplinary project integrates research findings into graduate and undergraduate courses and promotes STEM interest among high school students. The overarching goal of this project is to advance optimization theory and algorithms for robust machine learning model training by investigating their structured minimax optimization problems. The minimization problems in robust learning are structured based on the prediction function model, and we will investigate different types of models, such as neural networks. For each considered prediction function, the minimization component of the minimax problem is nonconvex; nonetheless, they each possess a distinct structure that we meticulously explore in separate thrusts. Moreover, within each thrust, we study different types of structured inner maximization problems motivated by the three applications closely examined in this project: distributionally robust learning, adversarially robust training, and discrete robust learning. The developed algorithms can significantly advance these application areas in terms of computational cost and accuracy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.