EAGER: Towards Adversarial Attack Resistant Machine Learning Systems

University Of Massachusetts Amherst, Amherst MA

Investigators

Abstract

Machine learning based pattern classification and related advances like deep learning have demonstrated impressive capabilities in multiple application domains, ranging from computer vision to medical diagnosis. However, it has also been shown that it is relatively straight-forward to create adversarial inputs that can fool machine learning models. The goal of this project is to develop defenses for machine learning models that are robust even in the face of sophisticated and determined adversaries. This project will have broad impact on the security of machine learning systems, advance cross-disciplinary research, and promote participation of undergraduates and under-represented groups in computer engineering research and education. This project will pursue two lines of defenses designed to hinder gradient ascent techniques used in adversarial input generation. The first line of defense will add controlled noise to output confidence levels to deny an adversary access to the precise classification boundary, while seeking to preserve model accuracy. The second line of defense will pursue choosing a random model in a query step from a pool of multiple trained models which have similar classification accuracy but differ in internal parameters and confidence levels. To test effectiveness of defenses, this project will also develop a gray-box model for accelerating adversarial input generation from a black-box machine learning model. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.