EAPSI: Using Prosody to Learn Phonetic Categories

Hitczenko Katarzyna A, Riverdale MD

Investigators

Abstract

Speech is highly variable. Though we often do not notice, every time we say a word or sound, it comes out differently. Our best language technologies (e.g. Siri) often fail precisely because of this variability, which shows just how difficult it is to learn about the sounds of language from variable speech. Nonetheless, infants learn about the sounds of their language within the first year of their life, and they do this with much less data and fewer resources than language technologies have access to. This project tests hypotheses about how infants solve this learning problem so effectively, by using computational models to simulate the learning process. This work will be done in collaboration with Dr. Reiko Mazuka at RIKEN Brain Science Institute, who has the only existing dataset with the necessary annotations for this project. Understanding how infants, the most successful language learners, learn can help improve language technologies so that they work more successfully across a wider range of languages and are more usable by a larger proportion of society, including people with accented speech and children, whose speech continues to pose problems for such systems. Various unsupervised sound category learning models will be tested on their ability to learn the Japanese vowel contrast, which no previous models have successfully learned. One of the factors that contributes to the difficulty of this particular learning problem is prosody - or the rhythm and intonation of speech - which systematically lengthens vowels in particular positions and causes overlap between short and long vowels through a process known as phrase-final lengthening. Knowing how prosody affects the duration of vowels and factoring out its influence could be helpful in learning the Japanese sound categories. This project will incorporate prosody into two existing computational models of sound category learning and see whether this improves learning outcomes. This award, under the East Asia and Pacific Summer Institutes program, supports summer research by a U.S. graduate student and is jointly funded by NSF and the Japan Society for the Promotion of Science.