Memory-Centric Resource Management for Data-Intensive Workloads on Clusters

College Of William And Mary, Williamsburg VA

Investigators

Abstract

Clusters of workstations have become standard system platforms for many scientific, commercial, and educational applications. This research focuses on effective usage of global memory resources to deal with dynamic job demands in large cluster systems. The targeted workloads are data-intensive scientific applications, Internet web accesses, and data processing for commercial databases. The first objective is to develop analytical/experimental performance models/tools to quantitatively examine the impact of the technology changes and data-intensive workloads to resource management policies, and to provide resource management guidance. The second objective is to design several memory-centric load sharing schemes by comprehensively considering dynamic job interactions and global cluster system resources. Finally, these schemes will be implemented and tested in a large cluster system. The impact and contributions of the proposed projects will be: (1) providing insights into memory systems performance and understanding potentials of memory-centric load sharing in clusters; (2) providing effective system solutions to adapt rapid changes of technology and workloads in cluster computing; and (3) making low-cost clusters more accessible for both scientists and business users to effectively run their large and demanding applications.