CAREER: Reliable and Accelerated Deep Neural Networks via Co-Design of Hardware and Algorithms

University Of Chicago, Chicago IL

Investigators

Abstract

Artificial intelligence (AI) systems are integral to a broad spectrum of applications, encompassing safety-critical and life-critical domains. However, hardware failures and design bugs have been observed during AI system deployment, leading to system malfunctions and potential consequences such as financial losses, reduced productivity, and even loss of human life. Furthermore, these issues directly impact hardware security and system sustainability. Existing solutions aimed at addressing these issues suffer from one or both of the following limitations: (1) high costs in terms of execution time, power consumption, hardware footprint, and/or data storage resources; and (2) limited coverage, as existing solutions are only capable of addressing a subset of these problems. This project will overcome these limitations through a fresh set of novel hardware-algorithm co-design approaches to simultaneously minimize costs and enhance coverage, advancing the state-of-the-art through an interdisciplinary combination of knowledge in computer architecture, robust system design, and machine learning. Successful completion of this project promises to mark a significant leap forward for AI systems, enabling them to be more efficient, reliable, trustworthy, and sustainable. Additionally, the project will enhance computer architecture education through creative visualization means and workshops especially targeting high-school students. The project also also places emphasis on facilitating technology transfer. This project encompasses three interconnected thrusts. The first thrust focuses on creating end-to-end approaches to fundamentally understand the impact of hardware failures and bugs on advanced deep learning workloads, mitigate these challenges through hardware-algorithm co-design, and incorporate a user study to explore adaptive architectural solutions tailored to individual users' needs. The second thrust targets the design of AI hardware for fine-grained mixed-precision deep neural networks. This involves creating a co-design framework to facilitate the co-evolution of hardware and software, optimizing accelerators for these networks, and simultaneously tailoring network models to these accelerators through a feedback loop, addressing susceptibility to design bugs. The last thrust explores an innovative approach for generating network parameters instead of storing them. The parameter generation algorithm will be integrated into training algorithms to optimize network accuracy and minimize area, power, and storage costs, while also addressing reliability and security threats posed by system memories. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.