Device-DS as a window into mechanisms of speech production: An investigation of adults and children

Cohn, Michelle Dana, Davis CA

Investigators

Abstract

This award was provided as part of NSF's Social, Behavioral and Economic Sciences Postdoctoral Research Fellowships (SPRF) program. The goal of the SPRF program is to prepare promising, early career doctoral-level scientists for scientific careers in academia, industry or private sector, and government. SPRF awards involve two years of training under the sponsorship of established scientists and encourage Postdoctoral Fellows to perform independent research. NSF seeks to promote the participation of scientists from all segments of the scientific community, including those from underrepresented groups, in its research programs and activities; the postdoctoral period is considered to be an important level of professional development in attaining this goal. Each Postdoctoral Fellow must address important scientific questions that advance their respective disciplinary fields. Under the sponsorship of Dr. Zhou Yu, Dr. Georgia Zellou, and Dr. Katharine Graf Estes at the University of California, Davis, this postdoctoral fellowship award supports an early career scientist exploring how individuals adjust their speech when talking to voice-activated artificially intelligent (voice-AI) devices, such as Amazon's Alexa or Apple's Siri, which are becoming ubiquitous features of everyday life. This project investigates the ways in which adults and children engage with voice-AI; currently little is known about how people adjust their speech when talking to these smart devices. Studying human-device interaction is important to examine the changing relationship between humans and technology; as digital devices become more prevalent, it is critical to understand their impact on our communicative and social behavior. This project explores the ways in which adults and children adapt their speech when talking to voice-activated digital assistants (e.g., Amazon's Alexa), compared to adult human interlocutors. This line of work provides a way to test differing theoretical predictions as to the extent that speech-register adjustments are driven by functional motives (e.g., intelligibility) and social factors (e.g., gender). For instance, this research explores whether the same functional motivations that apply when correcting comprehension errors to human interlocutors apply in device-directed speech (DS), such as in manipulating the phonological nature of errors, to carefully control the level of intelligibility-related pressures in communication. At the same time, this project explores how social factors may impact speech adaptation strategies, such as by interlocutor type, speaker age, or device gender. This project additionally involves important methodological innovations in programming and running experiments directly through a digital device platform. Overall, this project aims to fill a gap in our knowledge in the acoustic-phonetic adjustments humans make when talking to voice-AI devices, and can ultimately reveal the underlying mechanisms in speech production by different speakers (e.g., based on age, gender, device experience), contributing to basic science research. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.