CAREER: Physics-inspired Machine Learning with Sparse and Asynchronous p-bits

University Of California-Santa Barbara, Santa Barbara CA

Investigators

Abstract

Recent advances in the fields of Machine Learning and Artificial Intelligence (AI) have created practical applications ranging from powerful chatbots that can generate meaningful conversations or AI artists that can generate striking art. Behind the stage, however, there are enormous costs in energy, time and physical resources to train such AI models, making them costly, limiting accessibility and preventing democratized use. Moreover, these revolutionary advances have come at the worst possible time from a microelectronics viewpoint, since it has become significantly hard to improve the energy efficiency and performance of modern transistors whose dimensions have reached atomic dimensions. This project is about designing a new kind of physics-inspired and probabilistic computer, contrasting conventional fully-deterministic computers. The approach is to start from inherently noisy magnetic materials and devices to build probabilistic bits (p-bits). Networks of connected p-bits can then be suitably configured to efficiently solve computational problems encountered in probabilistic machine learning containing a large family of powerful algorithms that are hard to train in conventional computers. Because the underlying building blocks are naturally probabilistic in this approach, they can be used to implement probabilistic learning algorithms far more efficiently compared to conventional computers, where mimicking true randomness comes with high costs in area and energy consumption. The interdisciplinary nature of this project will require the synergy and rethinking of several different layers of the computing stack from devices, architectures and algorithms such that new types of energy-efficient, physics-inspired and probabilistic computers can be built to help with the greatest computing challenges of society. The specific approach of this CAREER project is to design physics-inspired probabilistic computers (p-computer) tailored for probabilistic machine learning algorithms. These p-computers will go beyond existing small-scale prototypes by combining magnetic nanodevices called stochastic Magnetic Tunnel Junctions with powerful CMOS-based field programmable gate arrays. The main aim will be to demonstrate the first large-scale demonstration of a CMOS + stochastic MTJ architecture for probabilistic computing where 10,000 digital p-bits will be augmented by 100 stochastic magnetic tunnel junction-based p-bits. Augmented by the true randomness and the asynchronous dynamics naturally provided by stochastic magnetic tunnel junctions, these heterogeneous processors are expected to provide orders of magnitude energy and performance improvement over optimized Graphical and Tensor Processing Units commonly used by present-day AI systems. The application of these p-computers to quantum and classical machine learning algorithms in physics-inspired, hardware-aware and sparse networks will lead to computational advantage and better energy efficiency, facilitating the eventual integration of million p-bit computers. The findings of this project will lead to the development of unique device models and algorithms, interdisciplinary courses and tutorials. These will be disseminated on nanoHUB and YouTube covering a diverse array of topics, including statistical mechanics, machine learning and quantum computing. Through partnerships with supporting institutions in academia and industry, this project will strongly contribute to the workforce training of the “technology maestros” of the future who are deep in one field but broad enough to connect to related areas. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.