NeTS: Small: Convertible Data Center Networks

William Marsh Rice University, Houston TX

Investigators

Abstract

Part 1: Increasingly, the main drivers for scientific discoveries and technological advances are data intensive computations that are performed in massive networked computing infrastructures called data centers. Such computations involve moving data between servers and processors across the data center network. It is important to be able to adapt the network to match changing and evolving network communication demands in order to keep data centers scalable, cost effective, and energy efficient. This project is aimed at solving that problem. It will develop a new approach for data center networking called 'network convertibility'. A convertible data center network transforms its structure dynamically and adapts to different network communication demands from data-intensive computations. In doing so, it can provide optimal performance for a wide range of applications without any manual network re-structuring, resulting in a more efficient data infrastructure to serve the many application areas that rely on data analytics. Part 2: This project will develop a network convertibility approach which leverages small port-count converter switches to convert topologies dynamically. The converter switches are strategically placed and finely integrated into the physical data center network design, and serve as the agents for convertibility that can fundamentally alter the network's properties such as average hop count, topological regularity, clustering properties, etc. to adapt to the application needs. The network convertibility approach has the potential to enable bandwidth to be moved to transmission hot spots in a fine-grained manner as needed. It also has the potential to achieve both easy implementation and high bandwidth performance by allowing the data center network to be organized/implemented as a hierarchical Clos network, but with the ability to convert it to a flatter and less hierarchical network like a random graph. The project will address multiple challenging sub-problems that are broadly classified into (1) building block designs, (2) convertible network designs, and (3) control algorithm designs. Given a convertible network design, algorithms will be developed to maximize the benefits of the available convertibility for different use cases. The project will also perform implementation and experimental evaluation on server clusters in the PI's lab.