SI2-SSI: Distributed Workflow Management Research and Software in Support of Science

University Of Southern California, Los Angeles CA

Investigators

Abstract

This award funds the enhancement of state-of-the-art workflow technologies and their promotion within a broad range of scientific domains. The overarching goal is to advance scientific discovery by providing scientists with tools that can manage computations on national cyberinfrastructure in a way that is reliable and scalable. The key technology supported by this award is the Pegasus Workflow Management System (Pegasus). This program of work includes the development, support, and maintenance of Pegasus. Pegasus allows users to declaratively describe their workflow, then makes a plan that maps this description onto the available execution resources and executes the plan. This approach is scalable, reliable, and supports applications running on campus resources, clouds, and national cyberinfrastructure. The work conducted under this award will 1) enhance the sustainability of the Pegasus software through the expanded adoption of sound software engineering practices and improved usability, 2) enhance core capabilities, especially in the area of data management, to meet user requirements and make Pegasus easier to integrate into end-to-end scientific environments, 3) promote the adoption of workflow management technologies within domain and computer sciences. Intellectual Merit: Pegasus WMS brings innovative and powerful frameworks to the desk of the scientist. Through close collaboration with a broad community of engaged users, experimentation in large-scale distributed computing is made possible. This experimentation supports the development of new scientific workflow management concepts, frameworks and technologies. The proposed work also supports scientific reproducibility by providing a workflow management system that integrates and automates data, metadata, and provenance management functions. Broader Impact: Pegasus WMS has been adopted by scientists from different domains and has been integrated into end-user environments such as workflow composition tools and portals. The program of outreach and education facilitated by this award will expand the impact of Pegasus through tutorials, workshops, meetings with potential users, and online materials. The proposed interface enhancements will allow more end-user environments to leverage Pegasus? capabilities and will extend the impact of Pegasus to a broader spectrum of users.