EAGER: SaTC-EDU: Multi-Level Attack and Defense Simulation Environment for Artificial Intelligence Education and Research

University Of California-Irvine, Irvine CA

Investigators

Abstract

Artificial intelligence (AI) techniques, particularly machine learning (ML), are increasingly integrated into safety- and security-critical applications such as autonomous vehicles and malware detection. However, research has shown AI techniques can be vulnerable to cyber-attacks such as adversarial perturbation and data poisoning, potentially leading to catastrophic outcomes when decisions made by AI systems are manipulated. Despite significant research efforts in this area, the research community has disproportionately focused on only a few domains, such as image recognition, and a few simple adversarial setups. Meanwhile more security-critical domains, such as malware detection, and a variety of adversarial models that more fully represent the real-world, have been ignored. Furthermore, it is difficult to compare, contrast, and characterize the different approaches to developing robust AI systems because of the fragmented nature of efforts in this area. This also creates challenges for education efforts in AI and cybersecurity. This project aims to address these urgent issues with synergistic efforts in AI, cybersecurity, and education that will produce significant research and societal impacts. First, the results of the project will promote public awareness of the issues and research around the robustness AI via the dissemination of tools and materials. Second, the project will democratize research progress in robust AI to application domains that are currently underserved, such as malware detection. Third, the project represents a concrete step towards fostering a workforce with skills in building robust and secure AI systems. The platform developed by this project will be integrated into undergraduate and graduate courses at the University of California Irvine and made publicly available to researchers and educators. The specific aim of the project is to address issues of research fragmentation in robust and secure AI. The project team will develop a new platform, called Maestro, to simulate adversarial machine learning tasks, covering a variety of adversarial capabilities (access to gradients, model weights, predictions, etc.) for both attacks and defenses, under a formal access-control framework. The Maestro platform will make it easier to implement, compare, and develop novel adversarial ML algorithms, settings, and applications that have not been sufficiently explored, including backdoor exploitation of natural language processing (NLP), stealthy adversarial malware generation, and security analysis of program embedding. The architecture of Maestro not only provides a useful framework to structure pedagogical materials in AI and cybersecurity, but also will be used to build course materials using active learning and gamification strategies. The latter will engage students while teaching them essential concepts about building reliable and robust AI systems. This project is supported by a special initiative of the Secure and Trustworthy Cyberspace (SaTC) program to foster new, previously unexplored, collaborations between the fields of cybersecurity, artificial intelligence, and education. The SaTC program aligns with the Federal Cybersecurity Research and Development Strategic Plan and the National Privacy Research Strategy to protect and preserve the growing social and economic benefits of cyber systems while ensuring security and privacy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.