CRII: RI: Immune-Inspired Learning Foundations of Neural Network General Robustness

Illinois Institute Of Technology, Chicago IL

Investigators

Abstract

Driven by rapid advances in neural networks (NNs), artificial intelligence has achieved remarkable success in many fields. However, small perturbations invisible to humans can be purposely added to inputs to cause NNs to make incorrect predictions. What is more, attackers can even customize different perturbation strategies to bypass existing NNs' learning methods and defenses. Thus, one open question is how to make NNs more robust to multiple types of such adversarial perturbations. Humans have a highly evolved immune system that can defend against multiple threats, even those never encountered before. Inspired by the powerful immune system, this project aims to infuse key immune system principles into NNs to improve their general robustness, that is, their capability to defend against multiple types of perturbations. The research outcomes will benefit fields that demand robust NNs, such as public health and autonomous driving. Furthermore, this project is planned to support cross-disciplinary education and research projects (involving machine learning and biology) for both undergraduate and graduate students, with outreach activities to high schools and particularly students from underrepresented groups. To reduce the substantial gap between existing machine-centric robust learning frameworks and robust immune models, this project focuses on incorporating into neural network design three robust immune-system components to help neural networks defend themselves against various attacks and continuously harden themselves. The proposed research consists of three aims. The first aim is to develop an immune-inspired population-point hybrid optimization that can effectively search for robust solutions and maintain the searching efficiency via a self-adversarial mode connectivity strategy. The developed technique will improve existing point-based learning approaches, which easily become trapped in bad local minima. The second aim considers neural network learning from a robust immune consensus perspective that incorporates stochasticity into learning, allowing NNs to capture global feature information. The third aim expands the first two aims by allowing NNs to adapt to unforeseen types of adversarial attacks with an immune-system-inspired lifelong learning regime consisting of a warm start defense strategy and knowledge distillation-based memory model update. This research effort will enable a new understanding of and a design framework for robust machine learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.