REU Site: BigDataX: From Theory to Practice in Big Data Computing at Extreme Scales

Illinois Institute Of Technology, Chicago IL

Investigators

Abstract

This award establishes a new Research Experiences for Undergraduates (REU) site at Illinois Institute of Technology and the University of Chicago. The site, which is named BigDataX, focuses on undergraduate research in both the theory and practice of big data computing at extreme scales. BigDataX includes a diverse group of 8 undergraduate students and 4 mentors spread out over the two institutions. Students will conduct research in the area of big data and how it will address issues related to the design, analysis, and implementation of run-time systems and storage systems to support big data applications. This work includes making extreme scale computing more tractable, touching every branch of computing in high-end computing and datacenters. These advancements will impact scientific discovery and economic development at the national level, and they will strengthen a wide range of research activities enabling efficient access, processing, storage, and sharing of valuable scientific data from many disciplines. The primary focus of this award is to promote a data-centric view of scientific and technical computing, at the intersection of distributed systems theory and practice. The project team has identified various data-intensive applications from many disciplines such as astronomy, bioinformatics, medical imaging, that demonstrate characteristics of big-data applications. This work focuses on the design, implementation, and optimization of runtime systems to support parallel programming systems for both Many-Task Computing and High-Performance Computing. The work centers on distributed scheduling, dynamic provisioning, improved fault tolerance, and support for heterogeneous computing. To better support big data applications, the team is exploring distributed file systems and improvements to a variety of critical components such as metadata management, I/O access pattern coalescing, distributed provenance as well as exploring novel interfaces into distributed storage systems. This work involves real applications, real data, and real testbeds ranging from small clusters at IIT and UChicago, supercomputers at Argonne National Laboratory, to the Amazon AWS cloud.