CAREER: Architecting a Hardware-Software Co-Designed Data Management System for Heterogeneous Memory Computers

University Of California-Davis, Davis CA

Investigators

Abstract

Data processing is integral to society, and the amount of data our computing systems are asked to process is growing rapidly. For example, some of today’s artificial intelligence (AI) and machine learning (ML) workloads have billions of parameters requiring terabytes of working memory for training and inference. To efficiently process this deluge of data, the data must first be moved from distant random-access memory to memory closer to the processor. In today’s systems, this data movement itself is one of the most expensive aspects of computation in terms of both performance and energy/power. To improve the efficiency of computing systems, this CAREER project aims to answer the question “where should data be placed and when should it move from one memory type to another?” The main insight driving this CAREER proposal is that bridging the semantic gap between the application which knows how the data is used and the hardware which directly interacts with the data will enable a new set of optimizations. Leveraging this insight, this CAREER project will develop a new interface for data movement which separates the data movement mechanism from the data movement policies. This separation of concerns will enable programmer hints and runtime-directed data movement which will significantly increase the performance and efficiency of heterogeneous memory systems. This CAREER project will improve the efficiency and performance of a wide variety of applications including AI/ML which increasingly depend on both high memory performance and large memory capacities. To broaden the impact, the techniques developed in this project will be realized in software, as a hardware-software codesign, and as hardened logic. Further, the software models, systems, and hardware prototypes will be released open source as they are developed. This CAREER project will also lay the foundation for a sustainable undergraduate research internship program at UC Davis. This program will give experience with open-source software development, career development opportunities, and hands-on hardware-software co-design experience. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.