SHF: Small: Lightweight Virtualization Driven Elastic Memory Management and Cluster Scheduling

University Of Colorado At Colorado Springs, Colorado Springs CO

Investigators

Abstract

Data-centers are evolving to host heterogeneous workloads on shared clusters to reduce the operational cost and achieve high resource utilization. However, it is challenging to schedule heterogeneous workloads with diverse resource requirements and performance constraints on heterogeneous hardware. Data parallel processing often suffers from interference and significant memory pressure, resulting in excessive garbage collection and out-of-memory errors that harm application performance and reliability. Cluster memory management and scheduling is still inefficient, leading to low utilization and poor multi-service support. Existing approaches either focus on application awareness or operating system awareness, thus are not well positioned to address the semantic gap between application run-times and the operating system. This project aims to improve application performance and cluster efficiency via lightweight virtualization-enabled elastic memory management and cluster scheduling. It combines system experimentation with rigorous design and analyses to improve performance and efficiency, and tackle memory pressure of data-parallel processing. Developed system software will be open-sourced, providing opportunities to foster a large ecosystem that spans system software providers and customers. Enabled by lightweight containers, cluster scheduling and the underlying operating system can cooperate synergistically, such that, the dynamic resource demand of an application can be exposed to the operating system, and the cluster memory manager and scheduler can be assisted with rich run-time information retrieved from performance counters and operating system. Towards this end, the project aims to devise a distributed memory manager for data-parallel programs that can survive from memory pressure and enable elastic cluster memory management with architecture-aware container placement, design a cooperative paging to improve performance of memory swapping by extending the current virtual memory reclaim mechanism in Linux kernel, enable memory over-commitment for elastic cluster scheduling with a new service that can detect and exploit the over-commitment opportunities, and design a multi-queue based distributed task scheduler to manage performance interference and hardware heterogeneity. The contributions include a library of developed mechanisms and open-source system software at cluster and kernel levels that can significantly improve cluster utilization and application performance. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.