SaTC: CORE: Small: Generalizing Adversarial Examples in Natural Language

University Of Virginia Main Campus, Charlottesville VA

Investigators

Abstract

Deep learning-based natural language processing (deep NLP) plays a crucial role in many security-critical domains, including advancing information understanding and analysis for healthcare, legal justice, e-commerce, and social media platforms. Consequently, it is essential to understand the robustness of deep NLP systems to adversarial attacks aimed at reducing their accuracy and security. To combat these attacks, this project introduces techniques to automatically evaluate and improve the adversarial robustness of deep NLP frameworks, as well as tools and datasets that can serve as useful community benchmarks and research resources. This topic is a new and exciting area that can contribute to multiple disciplines, including adversarial machine learning, natural language processing, and software testing; the project will support several graduate students in receiving advanced, interdisciplinary training in these areas. This award defines adversarial text examples as inputs to a deep NLP system that are maliciously designed to fool a predictive deep NLP model towards wrong predictions while simultaneously satisfying language-oriented constraints. The goal is to investigate the interplay between deep NLP and adversarial robustness in three dependent tasks. The first task is to build a comprehensive benchmark for generating adversarial text inputs across multiple NLP formulations. A library, TextAttack, will help researchers gauge their NLP models' robustness and provide a unified framework for attack designers to benchmark their attacks against the current state-of-the-art. The second task investigates the robustness of interpretation strategies in deep NLP and designs generalized adversarial text to reveal vulnerabilities in NLP interpretations. The third task adapts work from software testing to create criteria that define when an adequate set of adversarial text examples has been generated. In summary, this project studies how to evaluate the robustness of state-of-the-art NLP systems against an adversary and develop techniques to achieve both robust predictions and robust interpretations in deep NLP. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.