Low-power Neuromorphic Chip Architecture with in situ Deep Learning

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

As ultra-low-power sensing devices are being deployed expansively in all-pervasive wireless computing and consumer electronic products that now profoundly impact our everyday life, enormous amount of data are continuously amassed from these ubiquitous sensors. To exploit these data efficiently for elevating application-specific objectives, built-in intelligences are warranted in these sensor platforms to endow transitions from the traditional rule-based computing paradigm to emerging data-driven computing methods. Deep learning on multi-layered networks of neurons has provided such an opportunity to achieve intelligent computing through learning from the abundant data collected by sensing devices. How to make the current energy-intensive deep learning that mainly runs on clusters of general-purpose central processing units (CPUs) and graphics processing units (GPUs) amenable to various low-power systems remains to be a formidable challenge. This project envisages inventing hardware-friendly learning techniques and designing customized low-power deep learning hardware capable of real-time in-situ learning. The innovative hardware will be implemented on various low-power platforms to significantly accelerate the deployment of the nascent Internet-of-Things (IoT) technology. The low-power deep learning chip will enable energy-constraint systems to sense, process, organize, and utilize the data more intelligently. Useful information associated with the collected data can be extracted and exploited at the front end, thereby reducing the system response time and the energy consumption in wireless communications. In this research project, energy-efficient spiking neural networks (SNNs) will be used to construct deep learning networks. By leveraging the sparsity in multi-dimensional input data such as image, audio and video, the use of event-triggered SNNs can potentially result in significant energy savings. Building deep neural networks in SNNs also has the advantage of good scalability as CMOS technology advances beyond 10 nm. Through address-event representation, hundreds of sub-SNNs can be interconnected to build large SNNs solving disparate types of large-scale problems. Underlying difficulties owing to lack of effective learning algorithms in SNNs will be addressed by formulating the modulated spike-timing-dependent plasticity (STDP) learning rules. Through these bio-inspired on-line new learning rules, hardware-based SNNs can be designed for the event-triggered computation and deep learning. The neural hardware will be at first prototyped and validated on commercial FPGA boards, before realizing digitally by using nano-scale CMOS technology. Finally, in order to further reduce energy dissipation as warranted in ultra-low-power wearable systems, emergent memristor, i.e., analog resistive memory technology will be co-integrated into CMOS chip to mimic high-density artificial synapses. Variation-resilient architectures and algorithms will be developed to fully exploit the density of the memristor arrays, while tolerating their concomitant conductance variations due to several manufacturing limitations. Further, the integration of research and education will train future engineering workforce encompassing minority and female. Instructive materials developed under this project will be disseminated to research communities and practicing engineers by leveraging the NSF supported nanoHub repository.