Collaborative Research: RI: Small: Robust Deep Learning with Big Imbalanced Data

University Of Iowa, Iowa City IA

Investigators

Abstract

This project promotes the progress of science and technology development by advancing artificial intelligence (AI) through innovations in scalable and robust computational methods. AI, especially deep learning, has brought transformative impact in industries and quantum leaps in the quality of a wide range of everyday technologies including face recognition, speech recognition and machine translation. However, in order to accelerate the democratization of AI there are still many challenges to be addressed including data issues and model issues. This project seeks to advance AI by addressing one critical issue related to data; i.e., data imbalance. This happens when the collected data for training AI models does not have enough instances representing some property the models are trying to learn. For example, molecules with a certain antibacterial property would be far fewer than all possible molecules making predictions of antibacterial properties challenging. The goal of this project is to develop algorithms with theoretical guarantees to make AI learn more effectively from the big imbalanced data. This project will also contribute to training future professionals in AI and machine learning, including training high school students and under-represented undergraduates. This project investigates a broad family of robust losses for deep learning. The research activities include (i) developing scalable offline stochastic algorithms for solving non-decomposable robust losses that are formulated into min-max, min-min formulations; (ii) developing efficient online stochastic algorithms for solving a family of distributionally robust optimization problems that are cast into compositional optimization problems; (iii) developing effective strategies for training deep neural networks by solving the considered non-decomposable robust losses; (iv) establishing the underlying theory including optimization and statistical convergence of the proposed algorithms. The algorithms are being evaluated on big imbalanced data such as images, graphs, texts. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.