Adaptive dynamic coordination of damping controllers through deep reinforcement and transfer learning

University Of Tennessee Knoxville, Knoxville TN

Investigators

Abstract

In the last decades, global environmental pollution, concerns with fossil fuel reserves, and advances in technology have led to actions that are transforming the power grid. Over the years, several states have adopted renewable portfolio standards and goals to increase electricity production from renewable sources such as solar and wind power. This has resulted in new grid behaviors in response to disturbances in the system. This phenomenon has created concerns about an increased risk of sustained oscillations that can cause poor electric service quality and can even lead to blackouts. A search for new effective control systems has created a new breed of controllers. Specifically, the design of new controllers has been studied for wind turbines, energy storage systems, and other emergent components. However, with this massive presence of non-standard controllers and the existing controllers in conventional power plants, there is an urgent need for coordination that would enhance the combined effect of all the controllers to avoid conflicting interactions among them. Presently, there is no coordination of this magnitude in either actual systems or theoretical studies. The problem is challenging and it requires adaptability as the grid is permanently changing during its operation. This project will add intelligence to the grid, and through a real-time adaptable coordination, it will diminish possibilities of blackouts by mitigating the unwanted oscillations in the system. The proposed adaptive controller coordination has tremendous potential to enable the current and forthcoming power grid with superior dynamic performance and stability. This research will help increase awareness of the importance of the power grid for our nation including the benefits and challenges of renewable energy. Pre-college students and teachers will be exposed to engineering principles and practical applications through participation in outreach programs. This project will transform the conventional notion of controller coordination to make it suitable for real-time control as well as adaptive to disturbances and operating conditions. This project will seek controller coordinating signals that will minimize the system oscillation energy. The physical concept of oscillation energy not only allows avoiding the use of arbitrary objective functions, but also serves as a mechanism to weight the importance of the different oscillation modes without having to target in advance the most critical ones. As preliminary work, the PIs have derived an analytical procedure for on/off controller coordination using the time integral of the oscillation energy and its sensitivity with respect to the controller gains. As this procedure is based on the linearization of the state-space model, matrices, eigenvalues, and sensitivities must be either calculated promptly after a disturbance or collected previously off-line for the most representative operating points and disturbances. To overcome this drawback, and to make the coordination feasible and practical, a deep reinforcement learning (DRL) framework is proposed that would make the coordination not only adaptive but also more effective. In this regard, this DRL framework will allow exploring both discrete and continuous coordinating signals. While a discrete signal can be seen as a controller's on/off mechanism, a continuous signal can be understood as a quantity that can scale up/down the controller gains within a specified range. Furthermore, as the system can be subjected to extreme disturbances, this project proposes the use of transfer learning so the DRL training can be transferred even if there are topological changes or severe operational changes such as generator outages or load rejections. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.