CRII: CSR: Online Analysis of Disk I/O for Automatic Storage System Optimization

University Of Louisville Research Foundation Inc, Louisville KY

Investigators

Abstract

Today's critical applications, including genome analysis, climate simulations, drug discovery, space observation, and numerical simulations in computational chemistry and high-energy physics, are all data intensive in nature. Storage performance bottlenecks are major threats limiting the performance and scalability of data intensive applications. The goal of this project is to develop a general framework for self-optimizing parallel storage systems that can alleviate storage performance bottlenecks, and thus can have a considerable impact on society by accelerating the innovation process in a multitude of domains of science. The results will also advance the state of knowledge in storage systems by benefiting a wide range of parallel storage systems including disk arrays, key-value stores, and parallel/distributed file systems. Broader impacts include mentoring and training K-12 students through summer camps and promoting involvement of underrepresented students in science and engineering. This research develops novel, theoretically grounded, and experimentally validated methods for online detection and automatic elimination of disk I/O bottlenecks. Specific goals include the development of: (i) new online methods for continuously monitoring disk I/O requests and analyzing them efficiently, guided by data stream mining and social network analysis theory, and (ii) automatically-triggered self-optimization techniques guided by bin packing, graph coloring, and network flow theory, which can carefully plan an adaptive data layout to improve disk I/O performance. Experimentation and validation using software simulation and prototype implementation are performed by analyzing what is theoretically possible, what can be achieved in practice, and trying to close the gap between the two.