SHF: Small: Acceleration Using Smart Memory-on-Chip

Regents Of The University Of Michigan - Ann Arbor, Ann Arbor MI

Investigators

Abstract

Memory-on-Chip structures are used in almost all modern computer processing chips, both general-purpose as well as accelerators chips. They occupy a large fraction of the die area, for example, in Intel's server class Xeon processor devotes 35 MB just for its last-level cache, while Google's Tensor Processing Unit dedicates 24 MB for on-chip storage. Furthermore, a processor spends disproportionately large fraction of time and energy in moving data over its memory hierarchy, and in instruction processing, as compared to actual computation. To tackle these inefficiencies, this project proposes a novel idea: re-purpose the elements in memory structures and transform them into large data-parallel compute units. Outcomes of this research have the potential to accelerate computing operations involving heavy use of data sets. Data stored in on-chip memory arrays share wires (bit-lines) and signal sensing apparatus (senseamps). The research is grounded in the obeservation that arithmetic operations can be computed over these shared structures by augmenting a few gates to them. This in-SRAM computing technique is referred to as bit line computing. The preliminary work has demonstrated the potential and feasibility of smart memory-on-chip. This project will explore novel vertically integrated solutions that explore broad use of smart memory-on-chip. The research will develop new operation primitives, programming framework and compiler for smart memories, design neural computing architectures, investigate utilizing smart memories in Application Specific Integrated Circuits (ASICs) and reconfigurable Field Programmable Gate Arrays (FPGAs), and explore the applicability to emerging embedded nonvolatile technologies. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.