Collaborative Research: SHF: Medium: Spatial Multi-Tenant Neural Acceleration for Next Generation Datacenters

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Recent advances in Artificial Intelligence are transforming many aspects of human life such as e-commerce, medicine, transportation, and beyond. Datacenter networks are the foundation of modern online services. As the world is recovering from COVID-19, society is witnessing an increased reliance on online services and machine learning. This explosive growth has created an enormous demand for computation resources in datacenters. However, today's approaches are extremely costly and energy-inefficient. In fact, if the current systems continue to grow, datacenters will account for 14% of the total worldwide carbon emissions by 2040. This project aims to address this challenge using advanced resource-sharing techniques tailored for machine learning workloads. In particular, this award enables the network operators to maximize the utilization of network resources while achieving high quality of service experience for the users. This work sets out to explore the timely requirement of multi-tenancy for machine-learning acceleration through a new paradigm called dynamic architecture fission. There is a significant degree of underutilization when it comes to machine-learning accelerators that stem from the rigidity of architectures and their single-tenant nature. As such, there is an imminent need to rethink custom accelerator design and adoption in datacenters where cost-effective resource utilization replaces unnecessary resource cloning. Similar to the case of microprocessors, multi-tenant acceleration can open up a pathway that remedies resource replication and underutilization. Nonetheless, multi-tenancy has not been a primary factor in the design of machine-learning accelerators because of the race for higher speed, the recency of accelerator adoption in datacenters, and challenges associated with accelerator multi-tenancy. To that end, this project aims to explore spatial multi-tenancy as a new dimension in accelerator design to tackle resource underutilization in datacenters and bring forth cost-effective deployment of machine learning accelerators. This new dimension will significantly help reduce the slope of over-provisioning in datacenters to pave the way towards greener cloud computing. The proposed spatial multi-tenant acceleration of deep learning at scale can substantially improve the cost-effectiveness of next-generation datacenters. Given the increasing demand for deep-learning services and the carbon footprint of training and inference, this proposal will have a significant socioeconomic and environmental impact. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.