SHF: Small: Designing QoS-Aware MPI and File Systems Protocols for Emerging InfiniBand Clusters

Ohio State University Research Foundation -Do Not Use, Columbus OH

Investigators

Abstract

The emergence of affordable, high performance networking technology like InfiniBand is fueling the growth of high-end computing (HEC) systems. However, there are no schemes in these systems to provide a minimal Quality of Service (QoS) for the execution of parallel jobs by taking network-level contention into account. InfiniBand provides feature-rich QoS mechanisms. The research focuses on the following novel research directions: 1) How to take advantage of InfiniBand QoS mechanisms to design a QoS-aware MPI library? 2) How to take advantage of these mechanisms to design a QoS-aware parallel file system? 3) How to dynamically provide QoS by monitoring network traffic and making adjustments to virtual lane arbitration at InfiniBand's switch and adapter hardware? 4) How to design and establish job priority guidelines with the proposed QoS framework? and 5) What kind of performance, scalability, efficiency and productivity benefits can be achieved by this proposed QoS framework with petascale applications? The transformative impact of the research enables next generation InfiniBand clusters and applications to be QoS-aware and highly efficient in addition to delivering performance and scalability. The research has significant impact on the design, deployment, and utilization of next generation ultra-scale systems with QoS support.