AF: Small: Robust and Secure Learning

Stanford University, Stanford CA

Investigators

Abstract

Machine learning (ML) systems play an increasingly central role in society--from ubiquitous speech recognition systems, to navigation systems, product recommendation systems, and deployed learning systems across manufacturing, industry, and healthcare. The near future, with complex computer vision systems, self-driving cars, and ML driven medical care and patient monitoring, promises a nearly pervasive presence of ML systems in our society. Despite promising performance in idealized settings, current ML systems are often brittle--they are sensitive to slight changes in the input data, and often have weaknesses that can be easily exploited by a malicious adversary. Resolving these current shortcomings is a necessary step in ensuring the stability, safety, and security of a society that relies heavily on machine learning. The central goal of this project is to develop learning algorithms that are robust, and secure. These go beyond the traditional goal of developing learning algorithms that achieve high accuracy, and address the broad need for reliability and safety in critical deployed systems. As an extension of the research component of the project, the investigator will continue education and outreach efforts. These include disseminating the research publications and code produced by this project, continuing to develop new courses and teaching materials on data-centric algorithms, machine learning, and related topics, and organizing a semi-annual forum for the exchange of ideas between industry and academia. The research core of this project addresses the lack of robustness of current learning and optimization algorithms. This lack of robustness takes the following two distinct forms. First, current algorithms are sensitive to changes in even a very small portion of the data-set on which they are trained. Second, even when trained on legitimate data, the learned models are often susceptible to "adversarial examples" in the sense that for the vast majority of data points--even data points in the training set--a small adversarial perturbation of the data point in question will result in the model outputting a completely different label. The presence of these two types of fragility in current learning systems raises the possibility of vulnerabilities to two new sorts of security threats: 1) the threat that a portion of the training data is either extremely biased and unreliable, or worse--that it has been generated by an adversary whose goal is to mislead the machine learning system, and 2) the threat that deployed machine learning systems can be tricked via minute but carefully generated adversarial modifications in their test points--modifications that are essentially invisible to humans. The project seeks to address these two critical weaknesses of current systems, by : 1) developing new algorithms that are robust to the presence of significant fractions of arbitrary -- including adversarial -- data, which can be applied to a number of fundamental estimation, machine learning, and optimization tasks, and 2) developing a rigorous understanding of why certain training algorithms yield models that are inherently vulnerable to adversarial examples, and develop tools for reducing this vulnerability. Additionally, this project investigates the computational, and information theoretic aspects of robust and secure learning, including developing an understanding of any potential trade-offs, for example between the amount of training data and computation time, and robustness or security of the resulting trained model. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.