Collaborative Research: IIS Core: Small: World Values of Conversational AI and the Consequences for Human-AI Interaction

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

People increasingly converse with Artificial Intelligence (AI)-powered systems in their everyday lives, such as smart assistants on mobile phones or home speakers. However, those interactions could have unintended and potentially negative effects on people if the AI system expresses cultural values or beliefs they disagree with. Indeed, when humans interact with each other, cultural differences can lead to “culture clash,” which may trigger misunderstandings, mistrust, rejection, or declines in well-being. Thus, understanding to what extent culture clash may happen in human-AI interaction, and how people react to AI systems that express varied values, is an important question in making AI-based conversational systems that are acceptable, useful, and inclusive of people from different backgrounds. This project aims to investigate the ways values and beliefs embodied within conversational AI systems vary and whether diverging values between humans and AI can have effects similar to culture clash between humans. The team of researchers will develop: (1) methods for measuring cultural values in a variety of language models that form the basis for conversational AI systems; (2) insights into how prominent language models compare to human values across the world and across groups in the United States; and (3) empirical knowledge of the effects of mismatching values between humans and conversational AIs on a person's own values, trust, and subjective well-being, and the implications of these for larger society. The team will also release datasets for other researchers and deploy an online platform to allow the public to explore the values expressed by conversational AI systems, share their own experiences with culture clash in AI-based systems, and think about potential culture clashes and their consequences. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.