CHS: Small: Collaborative Research: Role-Based Norm Violation Response in Human-Robot Teams

Colorado School Of Mines, Golden CO

Investigators

Abstract

Robots may need to carefully decide when and how to reject commands given to them, if the actions required to carry out those commands are not morally permissible. Most previous work on this topic takes a norm-based ethical approach, where a robot would operate under a set of rules describing what states or actions are morally wrong, and use those rules to explain its actions. In contrast, this project explores a role-based perspective, in which the robot reasons about the relationships it holds with others, the roles it plays in those relationships, and whether the actions requested of it are benevolent with respect to those roles and relationships. Specifically, the researchers will develop a framework to allow robots to reason in this way and generate explanations of its actions based on this reasoning. The researchers will then explore how role-based and norm-based command rejections compare in terms of how they affect human-robot teamwork, and design algorithms to allow robots to automatically decide what type of rejection to generate based on their context. These algorithms and explanations will be evaluated in two very different contexts with different types of relationships, roles, and rules: with civilian undergraduates at the Colorado School of Mines, and with Air Force cadets at the US Air Force Academy. This work will not only increase robots' ability to behave ethically and act as good teammates, but will also advance moral philosophy by providing experimental evidence for the relative importance and effectiveness of different tenets of role-based moral philosophy. More formally, the goals of this research are to investigate context-sensitive tradeoffs between rule-based and role-based responses, and the representations and mechanisms needed to facilitate role-based responses. The research team will do this by identifying metrics to assess response acceptability, quality, and effectiveness; modeling the generation of role-based responses and selection between role-based and rule-based responses; conducting experiments to validate those models and responses; and using the results to articulate novel moral and philosophical arguments. The team will start with exploratory studies at each experimental site contrasting the effectiveness of different command rejection phrasings formed by crossing different Speech-Act Theoretic communication strategies paired with different moral philosophical backgrounds, with respect to (a) field-standard survey measures of trust, likability, mindfulness, workload, and norm strength; (b) qualitative analysis of video data; and (c) statistical linguistic analyses from the multimodal interaction community. The researchers will then develop a framework for robots to generate these responses that will involve formal computational model development of inter-team relations, roles, and actions; machine learning-based modeling of norm violation response strategy selection using features such as task context characterization, role-theoretic proposed action benevolence, and expected responses to the responses; and integration into the DIARC robot cognitive architecture. Based on this work, the team will both advance traditional moral philosophical arguments and develop a novel framework in which moral philosophical arguments are justified based on the effects of computational models on moral psychology. Overall, the project will lead to foundational, interdisciplinary knowledge into norm violation response, consisting of algorithms for selecting between norm violation responses grounded in different ethical theories, guidelines for and insights into the design of morally competent language capable robots, novel computational accounts of role-based robot ethics, and novel empirically informed moral philosophical arguments. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.