EAGER: Measuring Real World Application Performance on Next-Generation Computing Systems

Indiana University, Bloomington IN

Investigators

Abstract

The project proposes to create a benchmark suite using real-world scientific applications. The benchmark suite will be developed jointly with the High-Performance Group (HPG) of the Standard Performance Evaluation Corporation (SPEC), a non-profit organization with the goal of creating and maintaining standardized benchmarks to evaluate performance and energy efficiency for the newest generation of computing systems. The project will leverage SPEC infrastructure to ensure continuous maintenance of the benchmark suite, as well as support for result submission, review, and publication on the SPEC website. Furthermore, the project will use the published results of the benchmark suite to create a public ranking of High-Performance Computing (HPC) systems deployed across the world. Partnership with SPEC will ensure that the results of the project will be disseminated to industry. The creation of a new real-world application benchmark suite jointly with SPEC/HPG, as well as the development of performance metrics suitable for application benchmarks, will lead to improved HPC system design and enable the better understanding of how next-generation computing systems must evolve. In addition to evaluating HPC systems, the proposed application benchmark suite will also enable the comparative performance evaluation of novel software and hardware ecosystems. Moreover, the proposed work builds on NSF investment in domain science application development by including these applications in the suite. Importantly, the project will be an opportunity to bring the benchmark community together to develop and define performance and throughput metrics for real world scientific applications. Finally, the project will deliver a benchmark suite that will be freely available to non-profit organizations and sustained by SPEC. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.