S&AS: FND: A Stochastic Ethical Decision-Making Framework for Long-Term Autonomy

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

For robots to effectively collaborate with humans, they must be able to reason about the ethical consequences of their decisions and actions, incorporating societal values, rules, and conventions. Consider, for example, autonomous cars, which are envisioned to become commonplace in 5-10 years. Autonomous cars will provide many advantages for road safety, and they will offer mobility to those with physical challenges. For such cars to be successful, however, multiple ethical problems must be solved. For example, is it allowable for a car to break traffic laws as it is taking a wounded person to a hospital? What should a car do if it recognizes that an accident is unavoidable? How can a car reason in a way that is understandable and acceptable in human society (and courts of law)? Important ethical questions also arise in applications of robotics to areas such as military engagements, law enforcement, and healthcare. To address these issues, the research will provide ways for robots to reason and plan their tasks and make ethically sound decisions. These reasoning procedures will enable robots to learn and adapt over time as laws and social conventions change. The research approach is based on normative reasoning integrated with Markov Decision Process (MDP) planning to enable the creation of an ethical intelligent Physical System (IPS) that will be evaluated via human experiments. The research is innovative in that it will provide: (a) an integrated way of reasoning about task performance and ethical behavior in the context of long-term autonomy, where ethical rules can be changing and action consequences could be task and context dependent, (b) principled ways of evaluating consequences of robot actions, by considering and resolving conflicts between domain goals and normative goals, (c) criteria to determine norm priority in a flexible and context dependent manner, (d) methods to consider the whole life-cycle of norms, namely norm activation, deactivation, contradiction, violation and obsolescence.