Acceleration Techniques for Lower-Order Algorithms in Nonlinear Optimization

Louisiana State University, Baton Rouge LA

Investigators

Abstract

This project focuses on developing efficient innovative acceleration techniques and their underlying theories for the algorithms in nonlinear optimization. The acceleration techniques and algorithms developed in this project will have broad impact in many areas of computational science, including imaging/signal processing, optimal control, computer vision, petroleum engineering, topology optimization, and electronic structure computations. The algorithms developed in this research will be made publicly available on the web and will be applied in solving various computational problems. In addition, the student involved in this project will have excellent opportunities to participate in interdisciplinary research. The research will include developing subspace techniques for nonlinear conjugate gradient method and accelerated nonlinear conjugate gradient methods with theoretically guaranteed optimal global complexity. A framework of inexact alternating direction method of multipliers (ADMM) will also be developed, in which multiple steps are allowed to solve the subproblem to an adaptive accuracy, while still maintaining global convergence even when the problem has more than two blocks. The project will also study acceleration strategies for gradient based stochastic optimization. In particular, adaptive strategies for choosing sample points and extracting quasi-Newton information based on the obtained stochastic information will be explored. In addition, a novel dual active set approach will be developed for solving smooth large-scale nonlinear optimization. For example, in projection on polyhedra, an algorithm can be developed to approximately identify the active linear constraints, while an asymptotically faster algorithm can be used to compute a high accuracy solution.