CAREER: Automated Analysis and Design of Optimization Algorithms

Northeastern University, Boston MA

Investigators

Abstract

Iterative optimization algorithms lie at the heart of modern data-intensive applications such as machine learning, computer vision, and data science. Society has become increasingly reliant on such algorithms for commerce, transportation, healthcare, emergency response, and national security. Despite their critical role in society, algorithms are typically designed and tuned using insight from experts, extensive numerical simulations, and other heuristics. This research develops a more principled understanding and approach to algorithm design that automatically accounts for sensitivity to parameter choice, robustness to noise, and other sources of uncertainty. This approach enables algorithms to be engineered in a way that guarantees performance and safety, which is similar to how airplanes, skyscrapers, and computer hardware are built. Iterative algorithms may be viewed as dynamical systems with feedback. In gradient-based descent methods, for example, gradients are evaluated at each step and used to compute subsequent iterates. By treating algorithms as control systems, this research leverages tools from robust control (specifically: integral quadratic constraints, graphical methods, and semidefinite representation) to analyze and ultimately synthesize a variety of algorithms under different assumptions in an efficient, scalable, and systematic manner. This research also involves collaborative efforts in the areas of graph structure learning of gene regulatory networks and interactive machine learning, which serve to test and validate new algorithm designs.