PFI:AIR - TT: Automated Out-of-Core Execution of Parallel Message-Passing Applications

University Of Minnesota-Twin Cities, Minneapolis MN

Investigators

Abstract

This PFI: AIR Technology Translation project focuses on translating a software technology to fill the need for executing parallel applications on architectures with limited physical memory. This software technology, called BDMPI (Big Data Message Passing Interface), is important because it will allow approaches that rely on computational modeling and simulation to efficiently and economically solve very large problems on existing and upcoming computer systems, many of which are optimized for low power. This capability will positively impact many science & engineering disciplines, government, defense, commercial companies, and non-profit organizations. The project will result in a software prototype of BDMPI. The advantages of BDMPI over competing approaches are that (i) it allows existing parallel programs written in MPI (Message Passing Interface) to automatically switch to an efficient disk-based execution with no software re-engineering efforts, and (ii) it provides a flexible framework for developing disk-based distributed programs that can achieve levels of performance that are higher than leading competing approaches (e.g., Hadoop). This project addresses the following technology gap(s) as it translates from research discovery toward commercial application. It will expand BDMPI to support a large subset of MPI's standard API (Application Programming Interface), it will optimize its runtime system so that to reduce the overheads associated with disk-based execution, it will implement fault tolerance features, and it will optimize its runtime system for solid-state disks. In addition, the personnel involved in this project (graduate and undergraduate students), will receive innovation, entrepreneurship, and technology commercialization experiences through the Step-It-Up program that places students in part-time roles of supporting and performing commercialization efforts, their participation in an "Innovation Training" workshop, and by interacting with attorneys during patent application drafting.