CAREER: Towards a Game Theory of Safe Human-Centered Robotics

Princeton University, Princeton NJ

Investigators

Abstract

Future robotic systems promise to transform our homes, cities and roads, but they also raise critical questions about safety and reliability. What assurances should we, as a society, expect and demand from these new technologies? How can we tell an unfortunate accident from an unacceptable safety failure calling for a large-scale recall? From robotic caregivers to delivery drones and autonomous cars, robots' promise to society hinges on their ability to operate safely around people, yet current prototypes fall short of the safety assurances we all but take for granted in bridges, elevators, and airplanes. Unfortunately, traditional algorithms that once kept robots safe in factories now struggle with the complexity of the outside world, while machine learning approaches that thrive on big data are blindsided by safety-critical black swan events. This Faculty Early Career Development (CAREER) project aims to lay a new foundation for safe robot autonomy, enabling robots to continually prove that their course of action is safe with respect to any possible eventuality, however unlikely, out of a broad range of operating conditions (e.g., an autonomous car will avoid collisions even if a nearby cyclist suddenly falls into its path). These transparent, clear-cut guarantees will improve robots' reliability and facilitate regulatory oversight and public trust. To this end, the project's research activities are tightly interwoven with (i) a renewed robotics curriculum, from high school to graduate programs, preparing future generations of engineers to build safety into the core of robot autonomy, (ii) a series of workshops on Public Trust in Autonomous Systems spurring conversations about safety certification between industry and policy experts, and (iii) collaborations with key industry players equipped to mature and deploy the newly discovered techniques as well as to ensure the alignment of research progress with technological needs and regulatory standards. The project's outcomes will directly impact the domains of autonomous driving and mobile robots, and gained insights may be further extended beyond robotics to improve the safety of other AI technologies, boosting the ability of the United States and the international community to integrate intelligent systems safely and beneficially into society. The project investigates the rapid onboard construction of safety-certified robot motion strategies in the face of complex physical dynamics, close interactions with people, extreme low-probability events, and limited sensing capabilities. The project's key technical insight is to bring these challenges under a unifying analytical framework grounded in dynamic game theory. The formalism seeks to ensure safety under all admissible realizations of the robot's uncertainty, hypothesizing a zero-sum game against the worst-case turn of events at each moment. Undue conservativeness is avoided through two mechanisms: an Operational Design Domain (ODD) that clearly delimits the conditions under which the robot must operate, as specified by designers or regulators; and a precise treatment of information structure, which captures the robot and other agents' ability to acquire and adapt to new information over time. Focusing on legged robot walking and dense urban driving, investigators will explore the use of deep adversarial reinforcement learning (compatible with raw sensory inputs, black-box perception and prediction modules, and up to hundreds of state variables) to tractably synthesize approximate solutions to safety games for ODDs allowing external forces, changing terrain and visibility conditions, and hazardous or misleading human behavior. The research team will further investigate real-time robust planning and control approaches to refine these "untrusted" learned solutions into verified motion strategies guaranteed to preserve safety against the space of ODD-allowed scenarios. By separating synthesis and verification, this methodology provides a scalable, systematic mechanism to construct ODD-certified safety filters that continually monitor robots' candidate actions and intervene when deemed necessary to preempt potential catastrophic failures. A crucial property of the resulting framework is that restricting the robot's computational resources would only result in a more conservative safety assessment, and therefore potentially degraded task performance, but never in a safety violation. The technical approach will be rigorously validated on legged robot hardware, autonomous driving datasets and simulators, and in-lab user studies to ensure its effectiveness and practical applicability to the deployment of safety-critical autonomous systems. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.