SHF: Small: Enabling Efficient Context Switching and Effective Latency Hiding in GPUs

North Carolina State University, Raleigh NC

Investigators

Abstract

Graphics processing units (GPUs), initially designed for computer graphics, are becoming widely used for general purpose computing. This project addresses two important challenges in GPU computing. First, it investigates schemes to enable GPUs to be preempted efficiently, which is critical for GPUs to satisfy the quality of service (QOS) requirement in the cloud environment. Second, the project looks into approaches to significantly improve the latency hiding capability of GPUs. This interdisciplinary research has two practical uses, efficient preemption empowering GPUs as truly shared resource and effective latency hiding improving both the GPU performance and energy efficiency. Graduate student advising and industry collaboration are two key aspects of the project. The design philosophy of GPUs is to exploit very high degrees of data-level parallelism (DLP), expressed as thread-level parallelism (TLP), to hide long instruction latency. As a side effect, GPUs feature high amounts of on-chip resources to store the contexts or the architectural states of the large numbers of concurrent threads. The large contexts result in long latency for context switching, which makes it difficult for GPUs to be truly shared in cloud servers. This research project leverages the nature of the single-instruction multiple-thread (SIMT) execution model to drastically reduce and compress the GPU context size. Software and hardware approaches are integrated to enable instruction-level preemption for GPUs to meet the QOS requirements. Fast context switching is also used to switch out stalled threads and switch in new ones such that the otherwise idle computing resources can be utilized to provide much higher latency-hiding capability. It essentially achieves higher TLP on GPUs without enlarging their critical on-chip resources.