I-Corps: Algorithm-Hardware Co-Design for Large-Scale Machine Learning

Harvard University, Cambridge MA

Investigators

Abstract

The broader impact/commercial potential of this I-Corps project is the reduction of the economic and technical barriers preventing the adoption of current and future advancements in machine learning technologies. Advances in artificial intelligence (AI) have pushed machine learning models to be computationally larger and larger. Relatively few organizations currently have access to the most sophisticated models due to the necessary computational and memory requirements required to train and deploy such models. This lack of access results in high costs for researchers and businesses attempting to apply AI in applications such as robotics, natural language processing, drug discovery, and computer vision. The technology developed here reduces the size of the models and optimizes hardware so that large-scale models can be used on existing systems. Improved access to state-of-the-art models will accelerate adoption of AI in the economy and potentially drive more rapid improvements in cutting edge AI-based discoveries. This I-Corps project develops a combination of several innovations in the field of machine learning acceleration. The innovation is aimed at simultaneous optimization at the hardware and software levels. By targeting both levels simultaneously, the technology enables speeds which are not possible by combining existing technologies individually. The core technology combines model compression techniques, custom kernels and hardware utilization designs for acceleration, and cloud orchestration algorithms. Prior results have shown capabilities to significantly reduce model sizes and speed inference results. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.