CAREER: Structure Exploiting Multi-Agent Reinforcement Learning for Large Scale Networked Systems: Locality and Beyond

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This project aims to develop a suite of Multi-Agent Reinforcement Learning (MARL) algorithms for the control of networked systems, which are ubiquitous and play an indispensable role in advancing our modern society. Examples cut across a broad spectrum, including power/energy grid, transportation systems, networked robots, etc. The control and operation of such systems have long been a tremendous challenge, due to the complex interdependence between different components in the system, the increasing number of connected and interacting agents, and the increasing environmental uncertainties. The project will bring transformative change to how networked systems are controlled and operated, achieved by using MARL to design novel policies with order-of-magnitude improvement in efficiency and reliability under complex and uncertain environments. The intellectual merit focuses on a unique structure-exploiting approach for MARL. Most of the existing MARL approaches take a black-box view of the underlying system without utilizing the underlying structure and are known to have scalability and stability/safety issues. In contrast, real-world systems have rich structural properties that are readily available to exploit, e.g. the connectivity topology in the power grid. In light of the above structural properties, the project exploits the underlying structure to design scalable, stable, and safe MARL for large-scale networked systems. The proposed framework is not only theoretically sound but also widely applicable. To show the wide applicability of the approach, this project will demonstrate it on power systems and load balancing in multiserver systems. The broader impacts introduce a unique structure-exploiting perspective, that is machine learning should be integrated with the structural property of specific engineering applications. The project includes plans to broaden the impact of this perspective. Beyond the research itself, the project includes a synergistic education, diversity, and a broadening participation plan: (a) development of a new course at the intersection of machine learning and engineering networked systems; (b) K12 curriculum development and outreach activities through the CMU Gelfand Outreach program; and (c) undergraduates research through the SURF program. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.