CAREER: Reliable and Efficient Data Encoding for Extreme-Scale Simulation and Analysis

University Of Massachusetts Lowell, Lowell MA

Investigators

Abstract

Transformative research in science and engineering to address challenges of our time, such as designing new combustion systems, depends on progressively sophisticated computational models and simulations that operate on high performance computing systems. These simulations and analyses are increasingly constrained by the massive volumes of data that they must use, generate, and analyze. To manage this enormous amount of data, this project explores innovative mechanisms to optimize the performance of these simulations by reducing data movement and maximizing the use of computing power, while minimizing errors and information loss. Such performance improvements support NSF's mission to advance emerging, data-intensive science discovery and contribute to solving the world's most pressing and complex contemporary science and engineering problems. This project implements comprehensive outreach and education to train the next-generation of professional workers and researchers in the latest computing architectures and programming methodologies, and provides rich opportunities for student engagement, research, and employment. It leverages multiple campus and national resources and implements proven, research-based interventions to attract, retain, and educate female and underrepresented minority populations in computer engineering, which furthers the US national goal of increased participation in engineering. The research goal of this project is to adapt techniques and formats for compressing video data to the investigation of novel data encoding and decoding schemes to optimize data movement and computation in data-intensive simulation and analyses. Innovative new mechanisms have the potential to efficiently reduce the volume of data generated and transferred while also enabling rapid execution of various analysis kernels using compressed data, and permitting seamless scaling of their performance on current and future extreme-scale platforms. The research objectives are to investigate data encoding/decoding of scientific datasets and harness encoded data, employ and scale encoded datasets seamlessly within current extreme-scale scientific workflows, and optimize machine learning and data mining algorithms with the goal of maximizing the use of computing power while minimizing errors. These new mechanisms are applied to an evaluation framework and validated on multiple extreme-scale data-driven scientific applications, including climate, multiphysics, and fluid dynamics. This approach is expected to transform data representation and encoding while incurring minimal disturbance to existing applications, responding to the trends in hardware architecture and dataset characteristics. It is anticipated to improve the overall performance of computational scientists' workloads by reducing defensive and productive I/O costs, respectively, up to 100x and 200x data reduction spatially and temporally, potentially resulting in up to an overall 50x I/O cost improvement. The project leverages multiple collaborations in order to establish the governing principles for system co-design and scalable system software layers for better data encoding within world-class computational infrastructures. This project strengthens the University of Massachusetts Lowell computer engineering curriculum, broadens participation in computer engineering, and creates a collaborative, interdisciplinary research program geared toward exploiting ever-evolving computing paradigms. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.