CAREER: Data-Driven Network Resource Management Systems

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Modern networks require sophisticated systems and algorithms to manage resources efficiently and deliver high quality of experience to users. These systems are critical to services society has come to rely on, from video streaming to social networks to AI applications. Video streaming, for example, involves numerous systems that control everything from the resolution of the video to the network path and the video download speed based on dynamic network conditions. As networks and applications have become more complex, existing approaches have become inadequate and designing algorithms that deliver high performance in all conditions has become exceedingly difficult. The goal of this research is to address this challenge by developing network systems that learn to manage resources automatically through experience by applying new machine learning techniques. This new paradigm, if successful, will make networks simpler to design, more efficient and cost effective, and able to deliver better services to businesses and consumers. This project's goal is to develop the algorithmic and systems foundations for designing resource management systems that use modern reinforcement learning and other predictive control techniques to achieve strong performance across heterogeneous networks and applications. To this end, the researchers plan to build a series of practical systems for important applications, including schedulers for cluster computing systems (e.g., for data-parallel analytics workloads), and context-aware network control protocols (e.g., for adaptive streaming of 360 virtual reality video). In building these systems, the researchers will tackle fundamental challenges that confront data-driven network resource management, including (i) techniques to represent workloads (e.g., graph-structured jobs) and networks (e.g., topologies, queues, flows) to facilitate learning using neural networks; (ii) techniques to handle challenging resource management problems with large and deep action spaces; (iii) techniques to efficiently collect data across a myriad of devices for learning control models; and (iv) techniques to bootstrap learning models from data collected offline and continually train models safely after deployment This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.