SHF: Compute Caches: Opportunistic Parallelism in General Purpose Processors at Extreme Scale

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Computer designers have traditionally separated the role of storage and compute units. Memories and caches stored data. Processors' logic units computed them. It is not obvious that this separation is needed. A human brain does not separate the two so distinctly. This project addresses this fundamental question regarding the role of caches. Caches are used in almost all modern processors. They occupy a large fraction (over 70%) of the computer chip area. Latest Intel's server class Xeon processor, for instance, devotes several tens of megabytes just for its last-level cache. By avoiding movement of data in and out of memory arrays, this project will demonstrate the efficiency of compute caches, across a broad range of data-intensive applications that span several domains: cognitive computing, data analytics, security and graphs, and save vast amounts of energy spent in shuffling data between compute and memory units in modern computing systems. This project will develop novel SRAM array designs for supporting a rich set of operation types and address various architectural challenges that arise in integrating highly parallel compute caches with a general-purpose host processor. Until today, caches have served only as an intermediate low-latency storage unit. This project directly challenges this conventional design, and imposes a dual responsibility on caches: store and compute data. The key advantage of this approach is that it allows data stored across hundreds of memory arrays in caches to be operated on concurrently. The end result is that memory arrays morph into massive vector compute units that are potentially one to two orders of magnitude wider than a modern graphics processors (GPUs) vector units. This project considers vertically integrated solutions that cut across the computing stack: circuits, architecture, compilers, to applications. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.