SHF: Small: Energy Efficient Learning on Chip with Quantized Representations

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Machine learning models, especially deep neural networks, have been adopted in many real-time applications, such as speech and image recognition or object detection. These applications typically must be fast and energy efficient so they are usable in the field. This project develops quantized representations and algorithmic transformations for neural networks that represent large models with significantly less storage, energy and area, and with little or no accuracy degradation - in other words, enabling the use of efficient neural networks on mobile devices. The results of this project are poised to change how designers approach modeling, analysis, and optimization methodologies for large-scale deep neural networks, and more generally, any statistical learning applications that rely on expensive compute operations, with large memory footprint. The project will have not only a technical impact, but also an important educational and mentoring component by potentially changing how engineers are trained in a multidisciplinary fashion for dealing with next generation technological advances in general, and the problem of energy efficient machine learning in silicon in particular. The extensive experience of the Carnegie Mellon team in outreach to underrepresented groups involving training a diverse student body will be leveraged in this work, while further expanding the project's outreach to high-school and middle-school students. This project exploits the fact that using a quantized representation for on-chip training and inference can reduce the energy consumption and storage requirements of the associated hardware implementation. This is a crucial feature in achieving higher throughput and lower latency for real-time learning systems. This project addresses these challenges by developing novel quantization approaches for machine learning models that reduce both data movement and computation, and therefore reduce overall energy consumption. Furthermore, the research pursued herein relies on new algorithmic approaches for training quantized learning models so as to accelerate their training process without harming their accuracy, and learning quantized models on hardware (i.e., field programmable gate arrays) to verify their benefits and accelerate their adoption. To demonstrate feasibility, the project features a hardware accelerator-based testbed for inference and training models in a quantized fashion. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.