RI: Small: Towards a Formal Theory of Blameworthiness, Intention, and Moral Responsibility

Cornell University, Ithaca NY

Investigators

Abstract

As we move to an era of self-driving cars, autonomous drones, and robots helping people in work and everyday tasks, there is an increasing need to develop a theory of moral or ethical behavior. Policy-makers are already grappling with some of the issues. Jurisdictions around the world are beginning to formulate highway codes that take into consideration the presence of driverless cars. The goal of this project is to provide the foundations for a theory of blameworthiness, intention, and moral responsibility for such agents and applications. The study will evaluate this theory against the moral judgments that people make, and take the first steps to applying resulting policies in practice. A good theory will have to take into consideration tradeoffs among many competing and sometimes conflicting goals. This suggests that probability and utility will be involved. The theory will also have to deal with counterfactuals--that is, reasoning about what would have happened had circumstances or actions differed. Issues of causality will reveal whether an agent's action was responsible for an outcome, or if the action was coincidental to the outcome. Yet another relevant issue is determining intent: that is, determining whether an agent intended an outcome when performing an act. Pearl's structural-equations framework can model counterfactuals (interventions) and has been used as the basis of a definition of causality. In this project, the framework will be extended to capture degree of blameworthiness and intention. These definitions are given relative to an agent's epistemic state, which describes the agent's beliefs and preferences. Initially, the beliefs will be specified by a probability measure and the preferences by a utility function. The framework will then be extended to consider more qualitative representations of beliefs and preferences, to take into account issues of normality and typicality (e.g., to capture societal conventions), and to settings with multiple agents (e.g., multiple communicating driverless cars or robots), where game-theoretic concerns and issues of group responsibility vs. individual responsibility arise. The definitions developed under this framework will be tested to see if they capture people's intuitions about these issues, and will be applied to situations of practical interest. The theory will then be refined to take into account the outcomes of these experiments.