Optimizing MPI Programs through Compiled Communication

Florida State University, Tallahassee FL

Investigators

Abstract

The standardization of Message Passing Interface (MPI) has resulted in a large number of applications being developed using MPI. The current MPI standard hides network details and provides a simple interface for communication. While these features ease the task of parallel programming, they limit communication optimization opportunities. Specifically, with the current MPI, communication optimizations can be performed either by the compiler or in the library implementation. In both cases, optimizations are limited. Optimizations performed by the compiler must be architecture independent since the library hides architectural details from the compiler. Optimizations in the library implementation are usually architecture dependent. However, such optimizations can only be applied to each individual routine. In this research, we propose to optimize MPI programs through compiled communication. By simultaneously using the application communication information and the system architecture information to perform communication optimizations, compiled communication overcomes the limitations in the traditional communication system. We will accomplish the proposed work by developing systems that allow programmers to write regular portable MPI programs and enjoy the high communication performance achieved through compiled communication. The systems can also be used to optimize existing MPI programs. We will concentrate on two related parts of this research: extending the MPI standard by adding new routines to support compiled communication, and developing a restructuring compiler that analyzes an MPI program and transforms the program into one that employs the compiled communication technique. We will also develop techniques for applying compiled communication to irregular problems and investigate the interaction between the library and the compiler support for compiled communication.