DC: Medium: Designing and Programming a Low-Power Cluster Architecture for Data-Intensive Workloads

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This project defines a novel architecture for energy efficient computing, called FAWN, or a "Fast Array of Wimpy Nodes", which uses a large collection of small, slower computers to do tasks formerly handled by a single large, fast computer. By using more, slower, computers in tandem, the architecture runs more energy efficiently. However, doing so in a practical manner requires overcoming many significant challenges: The individual nodes have much less memory than programmers are used to; the computational task must be split across a much larger number of nodes, and the load must be balanced between them so that no single slow computer will cause the overall operation to be delayed too long. Modern datacenter environments further operate under very strict requirements for processing delay in order to ensure user satisfaction; these requirements may be harder to meet when running on the FAWN architecture. The project is tackling these challenges by developing new techniques for storing and managing large volumes of data on fast solid-state (Flash) memory. Through collaboration with experts in CS theory, the project is developing new algorithms for memory-efficient, highly parallel data analysis and data-mining, and implementing and evaluating them in a demanding, high-performance environment. Through a collaboration with Intel, the project is constructing a 100-node prototype of a FAWN cluster using cutting-edge low-power processors. This project has the potential to substantially reduce the capital and operating costs of large Internet services, and to greatly reduce their energy costs and footprint. It may also substantially advance the state of the art in practical algorithms and systems architectures for data-intensive and energy-efficient computing.