Partnership to Advance Throughput Computing (PATh)

University Of Wisconsin-Madison, Madison WI

Investigators

Abstract

The Partnership to Advance Throughput Computing (PATh) project will expand Distributed High Throughput Computing (dHTC) technologies and methodologies through innovation, translational effort, and large-scale adoption to advance the Science & Engineering goals of the broader community. The project will offer improved ways for scientists and administrators to manage computing workloads and resources – transforming raw capacity into effective capacity with High Throughput Computing (HTC) principles that maximize the sustained use of distributed computing resources toward computational problems. PATh’s distributed HTC approach will strive to increase the national return on investment of compute resources by enabling institutions to share computing capacity, simultaneously maximizing utilization while also giving smaller campuses easier access to this vital capacity. The techniques advanced by PATh serve the national interest by promoting the progress of science, from academic research groups, to large collaborations, institutions, and industry, with training and outreach components designed and executed to empower the communities that can benefit from these services. PATh is aligned with the NSF National CI Coordination Services blueprint’s goal of producing an “an agile, integrated, robust, trustworthy and sustainable CI ecosystem that drives new thinking and transformative discoveries in all areas of science and engineering (S&E) research and education.” PATh brings together two entities with a strong history of supporting dHTC-enabled research: the Center for High Throughput Computing (CHTC) and the Open Science Grid (OSG) Consortium. The team that founded CHTC at University of Wisconsin–Madison pioneered, in the mid 1990s, the concept and principles of HTC and has advanced and sustained HTCSS ever since. Roughly in parallel and driven by the needs of physics researchers, a diverse and multidisciplinary collaboration laid the foundation of the OSG in the late 1990s to develop, deploy, operate, and sustain a shared and global dHTC ecosystem of sites spanning geographic and administrative boundaries. PATh enables these two partners to align their efforts and leverage each other’s respective strengths, coordinated under the roof of one project. PATh will develop a Fabric of Capacity Services (FoCaS) to build vertical dHTC capabilities and drive the next phase of innovation in the HTCondor Software suite (HTCSS). The HTCSS offers a rich collection of tools for scheduling, federating compute and storage resources, and provisioning these resources to support high-throughput workloads at local, institutional, national, and international scales. In PATh, the HTCSS will receive new capabilities for incorporating external resources (particularly, allocations on HPC machines) into scientific workloads. In addition to the central role of the HTCSS in FoCaS production services, PATh’s core commitment will continue and fortify these services by integrating external technologies and ideas into the dHTC ecosystem. Building on the success of the OSG Connect service, and on campus-oriented Research Computing Facilitation methodologies pioneered by the CHTC at UW-Madison, PATh will create local dHTC expertise and capabilities at a growing number of campuses, including those under the NSF CC* program. Enhanced training, outreach, and community-building activities in PATh will further diversify the reach of dHTC impacts. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.