SPX: Collaborative Research: Global Address Programming with Accelerators

University Of California-Berkeley, Berkeley CA

Investigators

Abstract

Large-scale computing today is dominated by parallel computing, where a large task is divided into many smaller tasks and those smaller tasks run at the same time. Traditionally each of those tasks run independently up to a common stopping point, then they halt, exchange information, and continue. This global stop-and-communicate step is quite expensive. This project instead pursues a different approach, where individual tasks directly communicate with other tasks asynchronously, without having to wait for a global stopping point. This approach is likely to yield better performance on large-scale computing tasks, specifically on what is becoming the dominant large-scale machine, a heterogeneous machine with many CPUs and other many-core processors. The project will deliver a set of high-performance, open-source data structures and algorithm implementations to support irregular patterns of communication, notably those that arise in biology, graph analytics, and sparse linear algebra for machine learning. These will not only be directly useful for end users but also demonstrate how to design and engineer primitives for accelerator-equipped distributed-memory machines. The project also engages application developers (both in our groups and externally) to make the outcomes broadly useful. The project will develop a programming environment for accelerator-based HPC systems that integrates accelerators into a Partitioned Global Address Space (PGAS) model, which will allow direct communication between GPUs in a manner that is well suited to both applications and the underlying hardware. Specifically, GPU programming will be integrated with the UPC++ PGAS programming model ("GPUPC++"). The project will thus advance the state of the art in algorithms, programming models, and low-level support for the heterogeneous large-scale computers. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.