S&AS: FND: Uncertainty-Aware Safe Deep Reinforcement Learning

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Robot designers cannot always anticipate all real-world eventualities; hence robots will need to use algorithms that can learn to adapt to unexpected changes in their environment. However, if robots are to learn and adapt in the real world, they must adapt in ways that continue to maintain safety while learning. This project develops methods that ensure that robots continue to be safe even as they learn and adapt to unexpected changes in their environment. By enabling robots to be cognizant of their uncertainty, these methods can help robots operate more safely. Further, the methods will enable robots to be adaptive to unforeseen changes in their environment. These methods will be applied to autonomous driving, to enable robots to operate safely on surfaces with different levels of friction or on uneven terrain that might otherwise be unsafe to operate on. Such methods will enable safer autonomous vehicles for city driving in poor weather conditions, as well as for operating in off-road settings for search and rescue operations, patrol vehicles to detect animal poachers, and for other applications. This research develops a set of methods for uncertainty-aware safe robot learning. These methods will enable a robot to estimate the uncertainty of the effect of its actions; based on the estimated uncertainty, the robot will determine how to improve its performance while operating safely and cautiously. Furthermore, these methods will use the estimated uncertainty to determine how the robot should adapt its parameters to efficiently respond to environmental changes. The methods in this project will operate on complex policy classes such as those represented by neural networks trained with deep reinforcement learning. Such an approach will enable robots to achieve safe and adaptive long-term autonomy. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.