CAREER: An Agile Compiler Framework for Spatial Dataflow Accelerators

University Of Illinois At Urbana-Champaign, Urbana IL

Investigators

Abstract

In the era of large machine learning models, hardware accelerators play a key role in achieving fast and scalable training and inference performance. Among them, spatial dataflow accelerators (SDA), such as tensor processing units, have been extremely successful at accelerating demanding neural network workloads. Subsequently, many diverse accelerator platforms have been introduced with heavily programmable interfaces. In spite of these advances in hardware, compilers have been lagging behind, having only limited support for these accelerators. The current approach of manually constructing compiler backends is not sustainable with diverse and rapidly evolving accelerators. The project’s novelty is a set of automated and parameterized compiler construction methodologies that can generate optimized code targeting a wide array of spatial dataflow accelerator designs. The project’s impact is enabling hardware designers to rapidly build optimizing compilers for novel emerging architectures, which in turn will democratize the usage of new hardware platforms for accelerating diverse machine learning workloads. The investigator’s integrated education plan creates a novel machine learning compilers course that integrates formalisms, compilation techniques, and machine learning for compilers topics explored in this project. This new course offering prepares students with the necessary knowledge to succeed in careers that involve designing and maintaining compilers for hardware-accelerated machine learning workloads. The investigator plans to open source both the code and the data with public competitions, publish academic papers, collaborate with key industry partners with the possibility of technology transfers, hold academic workshops, and increase undergraduate participation to broaden the impact of the proposed research activities. The project explores novel automated compiler construction methodologies that are suitable for emerging SDAs. Compared to established commodity hardware platforms, emerging SDAs are more diverse, have faster design iteration cycles, and have expensive execution modalities. The project investigates novel techniques tackling three different aspects of backend code generation by synergistically innovating in both formal methods and machine learning fronts that can cater to the aforementioned characteristics of SDAs. First, it develops parametric representations and formalisms of tensor compiler intermediate representations (IR) and SDA descriptions. It uses these to automatically generate code generators specialized to each SDA. Second, the project develops innovative solutions that require significantly less target data to transfer learned cost models from mature accelerators. Finally, the project explores novel multi-fidelity optimization techniques that leverage different execution modalities to find faster auto-tuning solutions amidst expensive simulations. Successful completion of the project produces an agile compiler framework that can rapidly generate retargetable compiler backends for emerging spatial dataflow accelerators. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.