SHF: Small: Methods and Architectures for Optimization and Hardware Acceleration of Spiking Neural Networks

University Of California-Santa Barbara, Santa Barbara CA

Investigators

Abstract

Artificial intelligence is a powerful cross-cutting technology and is expected to promote broad advancements in science and technology as well as foster social benefits. To this end, exploring novel computational principles inspired by the brain may offer promising new avenues to enable artificial intelligence. This project is positioned to address key challenges in designing and engineering brain-inspired spiking neural models. As such, it may lead to methods, tools, and hardware system designs that will ultimately support new generations of software- and hardware-based artificial intelligence systems with potentially significantly improved performance and efficiency. This project will produce educational materials to be integrated into undergraduate- and graduate-level curricula on artificial intelligence and hardware system design, thereby providing workforce training opportunities in these areas of importance. The principal investigator will actively recruit undergraduate, underrepresented, and female students for research participation and training while partnering with various outreach programs. The results of this award may be derived in a variety of forms, including algorithms, software design tools, and hardware architectures and implementations that will be disseminated in broad research and industrial communities through publications, workshops, talks, and research collaborations. Engagement with US high-tech industries and other research organizations will be sought to broaden the impact of this work, promote potential technology transfer into practice, and offer additional mentoring and training of students under diverse industrial and research settings. Deep learning based on conventional non-spiking artificial neural networks (ANNs) has achieved great success in many application domains in recent years. Nevertheless, the conventional ANNs cannot immediately explore temporal codes and lack energy-efficient event-based processing. On the other hand, it is believed that attaining near-human-level intelligence requires computing paradigms that better mimic biological brains. As such, spiking neural networks (SNNs) offer a complementary biologically-plausible approach to facilitating future artificial intelligence systems. However, there are key roadblocks to a wider adoption of spiking neural networks. SNNs are much harder to train than conventional ANNs. There is a general lack of insights and systemic approaches for designing computationally-powerful SNNs, particularly SNNs with recurrent connections. Hardware acceleration of SNNs is hampered by complex data dependencies across both time and space, and unstructured firing sparsity. This work will start out by developing much needed accurate SNN training methods that can robustly learn precise temporal behavior and jointly tune spike count and spike timing. Scalable architectural design of recurrent SNNs and novel automated spiking neural structural optimization methods will be developed to support the design of computationally powerful SNNs. To enable energy-efficient high-throughput hardware acceleration, dedicated SNN hardware accelerator architectures that minimize expensive data movements and facilitate parallel processing in both space and time will be designed. Application-independent spike coding, spike compression, and architectures exploring unstructured firing sparsity will be investigated for SNN hardware acceleration. High-performance SNN hardware accelerators will be demonstrated on field-programmable gate-array devices. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.