EAGER: Collaborative 3D Materials Science Research in the Cloud

University Of California-Santa Barbara, Santa Barbara CA

Investigators

Abstract

The discovery and widespread implementation of advanced materials have long been challenged by the overwhelmingly complex combinations of elements and transformation paths that result in a plethora of material properties. Materials research projects generate datasets that range from a few tens of Gigabytes to terabytes, making it difficult to organize, share and analyze. This project brings new big data management and analysis techniques to materials research, giving scientists access to previously unwieldy datasets with new tools that accelerate the pace of innovation by allowing sharing of both complex data and analysis tools. The software will be deployed on the web, making it accessible to researchers worldwide to collaboratively explore the structure of materials. These tools and methods allow for the integration of experimental measurements in 3 dimensions (3-D) and 4 dimensions (4-D) with computational modeling of materials in extreme environments, such as aerospace engine components and the development of high efficiency thermoelectric materials. The project will push the boundary of large-scale web-based image and data analysis in multiple directions. First, the execution of complex scientific workflows on heterogeneous compute clusters will be simplified by exploiting virtualization techniques and modern cluster computing frameworks such as Apache Spark. We will compare overheads for parallel execution strategies on different frameworks for realistic workflows. Second, we will add provenance tracking and versioning for scientific workflows including a web-based browser that aids in making sense of past analysis runs and improves repeatability of experiments. We will extend graph query systems and graph visualization frameworks for this purpose. Third, we will integrate the capability to run Dream.3D pipelines in parallel across parameter ranges. This will enable the Materials Science community to rapidly explore effects of input parameters on the analysis results. The system will track sub-results and allow users to browse and query both metadata (e.g., "instrument name") and the complex output data in HDF tables. A new query system that spans modalities (tables, graphs, text, images) will be added. Towards achieving these goals, we will extend the existing BisQue image analysis platform that is widely used for large scale image informatics. The BisQue platform and the associated Materials Research tools will be distributed as open source.