SHF: Small: EVE: Ephemeral Vector Engines

Cornell University, Ithaca NY

Investigators

Abstract

Data-parallel kernels dominate the computational workload in a wide variety of demanding application domains, including graphics rendering, computer vision, audio processing, physical simulation, machine learning, and graph processing. Since it is no longer possible to rely on technology scaling for inevitable improvements in transistor performance and energy efficiency, there is renewed interest in specialized hardware to improve performance and efficiency compared to general-purpose processors for codes with significant amounts of data-level parallelism (DLP). Unfortunately, this specialized hardware lies idle when these computing systems are executing the many other interesting workloads that lack DLP. This project is exploring a new approach that is able to create specialized hardware "on-demand" by reconfiguring the memory already contained within modern computer systems to handle both storage and computation. The project's broader significance and importance are rooted in the need for computer architects to creatively mitigate the challenges imposed by the looming end of Moore's law, and the potential transformative impact of a software/hardware co-design approach. Two popular styles of hardware accelerators for exploiting DLP include data-parallel acceleration (DPA), which focuses on moving data in main memory to the compute hardware, and processing-in-memory (PIM), which focuses on moving compute hardware to the data in memory. In-situ processing-in-memory (PIM) is a recently proposed approach that attempts to significantly reduce the area overhead associated with exploiting data-level parallelism while at the same time reaping most of the benefit. In-situ PIM uses bit-line computation to perform basic bit-wise logical operations in a single read of a traditional memory array. Each memory column can be further transformed into a bit-serial ALU by adding extra logic, multiplexing, and state elements in the peripheral circuitry. This project makes two key observations about prior work on in-situ PIM: (1) in-situ PIM lacks compelling programming models; and (2) in-situ PIM requires massive parallelism to outweigh bit-serial execution overheads. This project is exploring ephemeral vector engines (EVE) as a new approach to address these challenges. EVE enables dynamically repurposing one or more private L2 cache ways to serve as on-demand (i.e., ephemeral) vector engines implemented using a novel reconfigurable bit-serial/bit-parallel in-situ processing-in-SRAM. EVE supports the unmodified RISC-V vector instruction set and can be rapidly reconfigured to use either bit-serial or bit-parallel execution. Bit-serial execution provides higher throughput but longer latencies, while bit-parallel execution provides lower throughput but shorter latencies. This project is using a vertically integrated research methodology spanning circuits, microarchitecture, architecture, and applications to explore three research thrusts. Thrust 1: EVE Circuits is exploring how to implement reconfigurable bit-serial/bit-parallel compute logic in the periphery of the SRAM array so as to minimize area, energy, and timing overhead. Thrust 2: EVE Microarchitecture is exploring how to efficiently map higher-level vector instructions into lower-level micro-operations. Thrust 3: EVE Architecture is exploring how to integrate the EVE microarchitecture into a complete system accelerating applications with regular DLP, while adding little to no area, energy, performance overhead for applications without regular DLP. This project is also pursuing two broader impact initiatives. The first is an ambitious yet concrete effort to increase participation of women in computer engineering by organizing a week-long computer engineering design experience for high-school girls. The second is an initiative to better preparing Cornell graduates for the post-Moore's law era, by integrating open-source CAD tools and the emerging PIM architectural approach into the undergraduate and graduate curriculum. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.