Collaborative Research: Data-driven Power Systems Control with Stability Guarantees

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

This NSF project aims to design a new data-driven power system control framework with stability guarantee. Power systems are experiencing a period of rapid changes due to the proliferation of renewable generation and distributed energy resources including solar, electric vehicles, and batteries. Many of these new technologies are interfaced with the grid through power electronic interfaces (i.e., inverters) that can be controlled at a much faster timescale compared to conventional machines. However, how to leverage such flexibility is nontrivial due to the nonlinearity, complexity, and uncertainty in the underlying power network. This project will bring transformative changes by developing new reinforcement learning (RL) algorithms for inverter-based frequency and voltage control with formal stability guarantees. The intellectual merits of the project include (i) a novel framework that bridges Lyapunov control theory and RL, therefore providing stability guarantee for learning-based controllers; (ii) neural network structure design that ensures stability constraint is met by design. The broader impacts of the project include various of new courses development and research opportunities for students interested in both energy systems and machine learning/AI. The proposed research consists of three thrusts. Thrust 1 focuses on developing the algorithmic framework that integrates RL with Lyapunov stability constraints, which serves as a foundation to later thrusts. Specifically, we will leverage analytical models to construct Lyapunov functions and engineer the structure of neural network-based controllers to meet the stability constraints. Thrust 2 uses machine learning to discover new Lyapunov functions for realistic power system models and design stable control policies. Thrust 3 integrates the theory and algorithms developed in Thrusts 1 and 2, and robustifies the controllers against modeling error, and network topology re-configurations in both transmission and distribution grids. The contributions of the project are two-folded. On the theoretical side, the proposed research bridges classic control and learning, where control theory provides the structural constraints that guarantee a controller is stable, and RL with neural networks searches over the large parametric spaces to find the best performing controllers that have this structure. On the practical side, our approach clears a critical hurdle in applying RL to power systems by guaranteeing the stability of the learned policy. We envision our framework will serve as the basis for future learning-based smart power system control architectures. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.