CAREER: Revamping the Memory Systems for Efficient Data Movement

Lehigh University, Bethlehem PA

Investigators

Abstract

Data movement efficiency is currently one of the most challenging impediments to the next performance leap in scientific computing and to the next qualitative improvement in big data analytics. The inefficiency of data movement is rooted in the conventional memory system design, which is based on the assumption that applications have good locality and is guided by the primary goal to overcome the memory latency wall. This design principle leads to a memory hierarchy design that progressively increases access granularity from the top to the bottom level. An access at one level of the memory hierarchy moves a contiguous block of data to another level to benefit from spatial locality. As the number of processor cores increases, however, good memory locality is difficult to achieve by programmer efforts or compiler optimizations alone, due to increasing contentions at all levels of the memory hierarchy and the dynamic nature of execution environments. The average utilization of a memory block is less than twelve percent even for highly optimized code, which results in a waste of energy, bandwidth, and on-chip storage. To improve data movement efficiency, this project seeks to revamp the memory systems to proactively create and redefine locality in hardware. This research holds the potential to fundamentally improve data movement efficiency through new memory system designs, which can motivate a complete rethinking of programming language, compiler, and run-time system designs as well. This project also seeks to train and mentor graduate and undergraduate students, promoting STEM among women and underrepresented groups, and developing outreach activities that raise awareness of the data movement efficiency problem among future programmers and computer engineers. The goal of this work is to re-architect the memory systems to improve data movement efficiency by exploiting fine-grain access correlations. The challenge is that tracking fine-grain correlations could require high meta data and control overheads. This work takes an end-to-end approach to share the meta data and control signals among different levels of the memory hierarchy. A new class of memory- and cache-architectures are developed in this research to redefine locality at each level of the memory hierarchy, which improves data movement, cache storage efficiency, and performance without introducing significant overheads. This research also investigates new memory organizations to improve data movement efficiencies in emerging memory systems such as non-volatile memory and near-memory processing systems. This work is evaluated using cycle-accurate architectural simulators and a set of important data-intensive workloads. Data structure- and algorithm- oriented optimizations are explored to allow existing and emerging workloads to take full advantage of the new memory architectures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.