SHF: Small: Ubiquitous and Transparent Near-data Computing for General Purpose Processors

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

As multicore processors scale, the overheads of data movement and communication become the primary limitations to achieving commensurate performance and energy-efficiency gains. This is especially true for “big-data” workloads that rely on enormous datasets. One increasingly popular solution is to perform computation near where the data is stored to avoid communication; this is a paradigm called near-data processing. Despite the potential, existing near-data systems have severe limitations: they often require programmer help, they are limited to support a small subset of workloads, and they do not exploit the full range of near-data processing technologies in a coherent framework. To address these challenges, this project develops new hardware/software abstractions that can enable near-data processing capabilities for general purpose architectures, and which can be constructed by a compiler or in hardware to avoid programmer burden. The potential impact of this research is to steer microprocessor design in novel ways that can help sustain expected exponential performance improvements, including efficiently scaling existing multicore processor size by an order-of-magnitude. In addition, our open source near-data compiler/simulation framework can foster research in this new direction. In terms of education, this project enhances courses with the infrastructure to give students cross-stack experience in co-designing hardware and software. Finally, the project develops an outreach program to provide mentorship and networking opportunities to prospective graduate students across multiple universities. Towards the goal of ubiquitous and transparent near-data processing, the primary technical insight of this research is that a more powerful near-data abstraction would encapsulate the program’s interaction with each individual data-structure: its address pattern, associated computation, and dependences -- this new program abstraction is called a “fiber”. Each fiber can be offloaded to a level of the memory hierarchy (e.g. last-level caches or memory) that best optimizes for data locality, and all accesses maintain sequential memory semantics. Fibers are attractive because their high-level properties can be used to determine an optimal near-data offloading strategy, and they are sufficiently coarse grain to enable efficient coherence and coordination. Towards these goals, the project addresses five fundamental intellectual questions: What set of primitive offloading strategies is required for generality, especially considering offloading to multiple memory-hierarchy levels? How is it possible to enable programmer and even ISA-transparent NDP? How to enable sequential memory semantics without being overwhelmed by coordination messages? How to perform near-data computation at high throughput and low overhead, either by reusing general cores, processing-using-memory (PUM), or reconfigurable accelerators? And finally, how to optimize data-placement for NDP, considering the importance of co-locating the operands of near-data tasks? Overall, this project explores a radical departure from the core-centric paradigm of traditional general purpose processors, and will develop new program abstractions and microarchitectures for efficient decentralized computation based on near-data principles. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.