CAREER: Developing an integrated model of speech entrainment

Cuny Brooklyn College, Brooklyn NY

Investigators

Abstract

Entrainment, the phenomenon of conversational partners' behaving more like each other in various ways, is an important feature for analyzing conversations. It is very common in human-human communication and is associated with high-quality conversations. Because more and more important conversations are happening between humans and virtual conversational agents, it would be useful for such an agent to be able to entrain on the person using it, to increase the rapport between them. However, there is still a lot that is still not known about entrainment, such as what causes it and why people entrain in different ways. The goal of this CAREER project is to learn more about entrainment in order to improve human-computer communication and to better understand human-human communication. This project will develop an integrated model of entrainment, the phenomenon of conversational partners? speech becoming more similar to each other, by analyzing the interactions and effect sizes of multiple factors known or hypothesized to affect entrainment, including speaker features such as gender, personality, and autism; interactional features such as liking and dominance; and utterance features such as dialog act or emotion. A corpus of spontaneous conversations will be collected to support the analysis. The methods used will include statistical and causal analysis, as well as a novel application of nonlinear machine learning methods to model the influence of each speaker on their interlocutor. The contribution of this project will be a deeper and more specific understanding of what entrainment can tell us about a conversation and how it should be modeled in a virtual conversational agent. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.