National Deep Inference Fabric for Very Large Language Models

Northeastern University, Boston MA

Investigators

Abstract

The National Deep Inference Fabric The National Deep Inference Fabric (NDIF) is a research computing project that will enable researchers to crack open the mysteries inside large-scale Artificial Intelligence (AI) systems. Modern large-scale AI systems such as Large Language Models have shown strong capabilities, passing a range of benchmark tests in mathematical, medical, and legal reasoning. However, because large-scale AI systems are trained automatically using massive amounts of data—instead of being designed line-by-line by a programmer—the internal workings of the current generation of AI are inscrutable to humans. Understanding how these systems work is an emerging science. But performing science on the internals of such large-scale AI systems requires substantial computational resources that are not practical at institutional scale, because a platform required to study the detailed computations of AI differs from the computing systems used for ordinary commercial deployment of AI. NDIF addresses this critical need by planning to develop a unique nationwide research computing fabric that enables scientists to perform transparent and reproducible experiments on the largest-scale open AI systems, in order to advance our nation’s understanding of their capabilities, as well as their limitations, robustness, safety issues, and impacts on human society. Led by Northeastern University in partnership with the NSF Delta AI high-performance computing cluster at the National Center for Supercomputing Applications, University of Illinois Urbana-Champaign, NDIF plans to consist of three major components. (1) A nationwide high-performance computing fabric hosting the largest open pretrained machine learning models for transparent deep inference. This National Deep Inference Fabric is a unique combination of GPU hardware with the creation of a deep network AI inference software to provide a remotely-accessible computing resource for scientists to perform detailed and reproducible experiments on large AI systems on the fabric. Once developed, the fabric will allow many scientists to efficiently and simultaneously share the same AI computing capacity to make efficient use of resources. (2) A novel open-source research software library to be created that enables scientists to develop and deploy new research methods on AI models by creating intervention code that inspects, modifies and customizes AI model computations. This library will help enables reproducible scientific experiments to be defined and executed on both the shared large-scale fabric and on a scientist’s own smaller-scale computers. (3) A nationwide training program to equip researchers and students in every part of the country to utilize NDIF to unlock critical research problems in every field impacted by large-scale AI. Developed together with the Public Interest Technology University Network, a consortium of 63 universities and colleges, the NDIF training program will consist of online modules, course materials, and in-person workshops hosted at multiple sites throughout the United States. It will create a network of experts in a range of fields impacted by AI, provide embedded expertise within their own institutions, and help create a next-generation workforce equipped to understand and harness the mechanisms and capabilities of the systems at the forefront of artificial intelligence. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.