CSR: Small: Energy Management for Heterogeneous MapReduce Data Centers

University Of Nebraska-Lincoln, Lincoln NE

Investigators

Abstract

Data-center-level energy management techniques have recently shown a lot of promise in improving data center energy efficiency. One challenge in developing such techniques is to support important types of workload. Current approaches, however, only consider managing compute-intensive applications. How to execute data-intensive parallel computations energy-efficiently remains a difficult open problem. World data is growing exponentially, doubling its size every three years. To facilitate large-scale data analysis and processing, a growing number of data centers start to support workloads that are managed with MapReduce-style frameworks. To support this increasingly popular workload energy-efficiently becomes very important. This project develops an energy management software system for heterogeneous MapReduce data centers. It is novel in several ways. First, it considers both computing energy and cooling energy and jointly minimizes their sum. Second, it develops feedback control algorithms to achieve energy-efficient utilization of multiple resources in heterogeneous data centers. Third, it develops aggressive consolidation techniques, matching the number of active nodes to the current needs of the workload. Novel consolidation-aware data management techniques are developed to make data placement and replication cooperative with server consolidation, saving energy while ensuring applications data availability and performance. If successful, this project will have significant impact on the society, by greatly conserving data center energy expenditures and corresponding carbon emissions footprint. Besides the technological impact, via outreach activities, curriculum development and matching minority students with this research, this project informs and educates the broader society and provides students hands-on experience in building energy-efficient computing systems.