NSF-SNSF: Scalable accelerated learning in continuous neuronal systems

Yale University, New Haven CT

Investigators

Abstract

Computer simulations are an essential part of modern scientific inquiry. They allow detailed investigations of phenomena at time and length scales difficult to observe directly. In particular for complex systems such as the human brain, they have become indispensable for testing hypotheses and making predictions for experiments. Cognitive phenomena such as learning and memory evolve on timescales from minutes to years, while the underlying processes on the level of individual nerve cells involve milliseconds timing. Bridging these timescales in computer simulations requires significant computational resources. This project develops a new hardware system specifically designed to meet the increasing computational demands arising for increasingly sophisticated multi-scale brain models. This new tool will both support fundamental neuroscience research and accelerate the translation of insights into biological computation into novel computing devices for commercially relevant applications. While the project focuses on computational neuroscience models, the implementation challenges and proposed solutions are applicable to the simulation of a broad class of dynamical systems such as power grids, financial markets, and epidemics. Finally, this project will support the continuous development of open-source workflows for the design of asynchronous digital hardware. Despite tremendous progress in recent decades, the biophysical mechanisms and computational principles underlying memory and learning in the brain remain poorly understood. Due to the complexity of biological systems and the difficulty in reproducibility (in the engineering sense) of results, researchers rely on computational models to test hypotheses and to make experimental predictions. Modern models of neuronal computation and learning are formulated in continuous-time with continuous interactions, i.e., as systems of ordinary differential equations. Yet, these are a poor fit for existing accelerators which are optimized for either continuous time with discrete (spiking) interactions or discrete time with continuous interactions. A significant implementation challenge on the path to a scalable accelerator lies in efficient parallelization of the spatially discrete, non-locally and non-uniformly coupled systems of equations, in particular due to the communication requirements arising from continuous interactions in continuous time. To overcome these challenges, we propose a convergent simulator architecture that can accelerate continuous-time models of learning using neuromorphic hardware principles. On the algorithmic end, we exploit the insight that continuously interacting systems do not necessarily require dense communication, i.e., that communication in implementations of these dynamics can be both sparse and adaptive to computational demand. The resulting algorithms are ideally suited for implementation in asynchronous digital hardware with fine-grained parallelism and share similarities with existing neuromorphic computing approaches. To validate our approach and ensure its usefulness to the scientific community, we create both an FPGA implementation and silicon prototype for a recently proposed cortical learning model and develop novel model variants for processing and learning across long time scales. This collaborative U.S.-Swiss project is supported by the U.S. National Science Foundation (NSF) and the Swiss National Science Foundation (SNSF), where NSF funds the U.S. investigator and SNSF funds the partners in Switzerland. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.