EAGER: Multi-Domain, Workflow Driven Computation System for Microbial Ecology Research and Analysis

University Of California-San Diego, La Jolla CA

Investigators

Abstract

This EAGER award to the University of California - San Diego, explores innovations to The Community Cyberinfrastructure for Advanced Microbial Ecology Research and Analysis, (CAMERA, http://camera.calit2.net/). CAMERA is a widely used infrastructure that provides a single software system for depositing, locating, analyzing, visualization and sharing data about microbial biology. A heavily utilized and core component of CAMERA is the workflow driven computational system for data analysis. The award anticipates analysis needed for next generation sequence (NGS) data. BLASTx Work flows using CAMERA's Refseq Protein database) will require nearly 2.5 Million core hours to complete analysis of a typical NGS data set, making it impractical and costly to rely on a single computational resource for an entire workflow. The PI will prototype novel heuristic-based dynamic resource scheduling approaches to allowing distribution of workflow tasks among heterogeneous computing domains such as local systems, a variety of commercial clouds, and multiple national resources. This research explores development of a multi-domain scheduling system where the individual component tasks of a workflow are not bound to a single computational resource or domain, but rather can be scheduled across multiple resources as well as across institutional domains. It benefits a rapidly growing segment of the scientific community in both large and modest laboratories whose instruments now generate massive amounts of genomic data at modest expense but who have insufficient access to resources needed for processing or analyzing these data. The cyberinfrastructure to be established will potentially be usable by other science communities facing similar computational challenges.