CRII: SHF: A Compiler and Runtime Infrastructure for Flexible Scheduling and Scheduling-Enabled Optimizations on GPUs

Colorado School Of Mines, Golden CO

Investigators

Abstract

Title: CRII:SHF: A Compiler and Runtime Infrastructure for Flexible Scheduling and Scheduling-Enabled Optimizations on GPUs The computing power of a GPU (Graphics Processing Unit) lies in its abundant memory bandwidth and massive parallelism. However, its hardware thread schedulers, despite being able to quickly distribute computation to processors, often fail to capitalize on program characteristics effectively, achieving only a fraction of the GPUs' full potential. Moreover, current GPUs do not allow programmers or compilers to control thread scheduling, forfeiting important optimization opportunities at the program level. This research aims to develop a new software-level infrastructure for flexible scheduling and scheduling-enabled optimizations on GPUs. The intellectual merits of the research are two-fold: 1) It develops compiler techniques to circumvent the restrictions from the hardware thread scheduler, which enable programmers or the runtime to flexibly schedule tasks to the GPU processors; 2) It designs runtime optimizations to leverage the flexible scheduling. The project's broader significance and importance are that it provides essential support enhancing the computing efficiency of data-intensive applications in the era of GPU computing and, due to the importance of these applications, fosters sustained advances in science, engineering, humanity, and health. The project designs a code transformation component to enable flexible scheduling. The transformation, named SM (Streaming Multiprocessor)-centric transformation, consists of two techniques. The first technique is SM-centric task selection, which breaks the mapping between tasks and thread blocks and directly associates tasks with processors. The second technique is a filling and retreating scheme, which addresses some behaviors of the hardware scheduler and flexibly controls the number of active tasks for each processor. The project also designs three types of optimizations, namely parallelism control, affinity-based scheduling, and processor partitioning, which leverage the scheduling support to optimize for parallelism, locality, and resource allocation. The project develops both static and dynamic approaches to efficiently searching for the optimal scheduling strategies adapted to address various program and input features.