CRII: SHF: Efficiency-Aware Robust Implementation of Neural Networks with Algorithm-Hardware Co-design

Yale University, New Haven CT

Investigators

Abstract

With the advent of Internet-of-Things and the necessity to enable intelligence in embedded devices like mobile phones, wearables etc., low-power and secure hardware implementation of neural networks is vital. Despite achieving high performance and unprecedented classification accuracies on a variety of perception tasks, Deep Neural Networks (DNNs) have been shown to be adversarially vulnerable. For example, a DNN can be easily fooled into mis-classifying an input with slight changes of image-pixel intensities. This vulnerability severely limits the deployment and its use in safety-critical real-world tasks such as self-driving cars, malware detection, healthcare monitoring systems etc. This project investigates hardware aware techniques to resolve or resist software vulnerabilities (specifically, adversarial attacks) by exploring the design space of energy-accuracy-robustness trade-off cohesively with algorithm-hardware co-design to create functional intelligent systems. Thus, the project seeks to develop robustness-aware algorithms broadly applicable to the energy-efficient and secure implementation of DNN engines on both current CMOS accelerator platforms and emerging memory technologies. Furthermore, the research will support the interdisciplinary development of a diverse cohort of PhD and undergraduate students, and the development of a graduate-level course at Yale University on neural network architectures and learning algorithms tied with robustness from circuit and system design perspective. The technical aims of this project are divided into two thrusts. The first thrust develops robustness centred algorithms in DNNs where techniques such as quantization, pruning among others are used to improve the adversarial resilience of models while yielding energy-efficiency benefits. This part also identifies a new form of noise stability for DNNs, i.e., the sensitivity of each layer’s computation to adversarial noise. This allows for a principled way of applying layer-specific algorithmic modifications that incurs adversarial robustness as well as energy-efficiency with minimal loss in accuracy. The second thrust benchmarks and implements the proposed robust computing models on emerging technology-based memristor crossbar-array platforms to investigate the hardware-level benefits (while comparing with standard CMOS accelerator baselines). In particular, design issues and complexities for implementing variable precision, stochastic and combined stochastic-deterministic neuronal activity will be investigated. The two thrusts offer a fundamental co-design infrastructure where algorithmic innovations will be used to optimize robust and efficient hardware implementations for neural networks. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.