Doctoral Dissertation Research: Mechanisms of adaptive plasticity in speech perception

Carnegie Mellon University, Pittsburgh PA

Investigators

Abstract

Comprehension suffers when a conversation partner produces speech that departs from expectations. A strong foreign accent, a distinct dialect, or even a stuffy nose can impact the precise realization of speech. Human speech perception is remarkably flexible. Although comprehension initially suffers when speech departs from the norms of a listener’s language-community, with exposure listeners rapidly adapt and comprehension rebounds. Understanding the means by which the human brain is able to flexibly map speech to language representations promises to be important in advancing next generation linguistic theories, in building more robust machine speech recognition, and in supporting individuals communicating in non-native educational contexts. This dissertation research project will simulate an accent by manipulating short-term speech input regularities to determine how speech perception adapts. This offers the advantage of providing a detailed measure of how the weighting of specific acoustic speech dimensions shifts as a function of short-term speech input changes such as accent. The central hypothesis of the research, supported by preliminary studies, is that adaptive plasticity in speech communication is driven by error-driven supervised learning. Study 1 tests a prediction arising from this hypothesis: when short-term speech input regularities depart from long-term norms, supervised error-driven learning supported by phonetic category activation will result in down-weighting of secondary acoustic dimensions available for speech recognition. Study 2 pursues this prediction via a neural network that implements the algorithmic theory. Study 3 addresses whether overt phonetic category decisions are necessary for adaptive plasticity in a naturalistic listening environment without overt phonetic category decisions. Finally, Study 4 uses scalp electrophysiology to examine the level of linguistic representation impacted by adaptive plasticity in dimension-based statistical learning. In all, the project will advance understanding of how the linguistic system tackles the challenge of communication when speech input deviates from the long-term norms that established linguistic representations. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.