CAREER: Demystifying Deep Machine Learning Models using Convex Optimization for Reliable AI

Stanford University, Stanford CA

Investigators

Abstract

This project develops a theoretical framework for deep neural networks, a type of machine learning model that has had tremendous success in a range of applications including image and speech recognition, robotics, and automation. Although these models have been successful, there remain significant open questions in the understanding of how they make decisions or how they can be made more efficient, robust and reliable. Additionally, there is a lack of transparency in the inner workings of deep neural networks, which can make it difficult to trust their output and interpret their results. By developing a theoretical framework to study and train these models based on convexity, which is a well-studied mathematical concept in optimization theory, this project aims to improve their reliability and interpretability, ultimately leading to more efficient and trustworthy artificial intelligence systems. This project likewise seeks to educate the next generation of researchers, and benefit society by enabling safe and effective applications of artificial intelligence. The technical approach is based on a novel convex analytic framework to study, train, and validate non-convex models, including deep neural networks. By leveraging the hidden convex optimization landscape in non-convex training losses, this project develops a theoretical foundation that should demystify the optimization and generalization properties of these models. By applying techniques from signal processing, compressed sensing and convex optimization, the project seeks to unify ideas and methods from diverse fields in order to advance the state of the art in non-convex models. This will advance our understanding of the fundamental behavior of deep neural networks and mitigate challenges associated with their use. In addition, this project will investigate the interpretability, verifiability and robustness of these models through the lens of convex optimization in practical applications. The diverse applications of neural networks can attract graduate students and contribute to the integration of modern deep learning topics in signal processing courses. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.