A Common Prosody Platform for Testing Theories and Models of Speech Prosody

Haskins Laboratories, Inc., New Haven CT

Investigators

Abstract

Prosody plays an important role in the complex mapping between meaning and the spoken expression of words, phrases and sentences; it is critical to our understanding of human communication and to our ability to exploit prosodic information in both automated language production and meaning recognition. Prosody research has seen significant development in recent decades, and numerous theories and computational models have been proposed. However, many fundamental issues remain unresolved and some are still under heated debate. This lack of consensus has resulted in slow advances in developing speech applications with capabilities for processing prosody. This project is a collaborative effort to accelerate progress in prosody research by developing a Common Prosody Platform (CPP). CPP will consist of an open-access web site that hosts a collection of trainable models in the form of Praat scripts, each implementing a major theory of prosody. All the scripts will consist of A) a computational realization of the basic assumptions of a particular theory, which is capable of generating surface prosodic forms (initially, F0 and duration), and B) a set of learning algorithms for automatic analysis-by-synthesis and global optimization, which allow the script to be trained on any speech corpora marked up with theory-specific categories. CPP will therefore facilitate theory evaluation by enabling them to make numerical predictions that can be directly compared with natural prosody in fine detail. CPP will be tested on autosegmental-metrical (AM) theory, Parallel Encoding and Target Approximation (PENTA) model, articulatory phonology/task dynamic model (TADA), and command response (Fujisaki) model, and apply them to English, Greek, Mandarin and Itunyoso Trique (an endangered tone language). The development of CPP will help bridge the current gaps between theoretical conceptualization, empirical investigation and computational modeling. The computational nature of the resulting trainable models will also make them readily transferable to applied areas, including speech technology, language teaching and speech communication disorders. The research approach developed in this project may also be extendable to a general paradigm in speech research, namely, theory testing by computational modeling.