ERI: Improving the Learning Efficiency of Adaptive Optimal Control Systems in Information-Limited Environments

Florida Institute Of Technology, Melbourne FL

Investigators

Abstract

This Engineering Research Initiation (ERI) grant will fund research that enables efficient, on-the-fly learning of optimal control strategies for complex engineering systems operating in uncertain environments, with application to connected and autonomous vehicles, thereby promoting the progress of science and advancing the national prosperity and welfare. Many emerging control systems in the artificial intelligence, automotive, robotic, and energy fields require that optimal actions be identified and executed across a network of components in the absence of detailed system knowledge and based on limited input data. Learning-based approaches have been developed to meet this requirement, but are challenged by very slow rates of learning, restrictive requirements on the control policy used to initiate the learning process, and complications due to sensor limitations and sparse data sharing between individual components. This project will overcome these challenges by building a new learning-efficient control framework that integrates advantages of existing methods and demonstrates new solutions for handling missing data streams and optimizing the communication structure between system components. When applied to networks of autonomous vehicles in complex traffic scenarios, the framework may enable improvements in roadway safety and reduction in road fatalities. The broader impacts of this project include outreach efforts to the public intended to show how artificial intelligence and automatic control can be safely leveraged, as well as training and preparation of undergraduate and graduate students to pursue further education and advanced STEM careers. This research aims to make fundamental contributions to the development of a learning-efficient, adaptive optimal control framework for nonlinear dynamical systems with completely unknown system models and under conditions of partial observability, and to enable the application of this framework to networked control systems with nontrivial communication topologies. It will achieve this outcome by developing a new hybrid iterative form of reinforcement learning that achieves a quadratic rate of convergence even if a system model and an initial admissible control policy are unavailable. Inspired by ideas from hierarchical reinforcement learning, a two-layer learning-efficient method will be created to enable simultaneous learning of robust distributed control strategies for individual network agents and an optimal network communication topology, including in the presence of communication delays between agents. Micro-traffic simulations and physical experiments will be used to test the theoretical framework in the context of collision avoidance in several formation control scenarios. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.