ITR: Modeling Degree of Articulation for Speech Synthesis

Oregon Health & Science University, Portland OR

Investigators

Abstract

The automatic conversion of text to speech provides a means to achieve universal access to on-line information. However, except for simple messages, speech generated by current synthesizers is both unpleasant and hard to understand: even though words presented individually are quite intelligible, listeners are generally unable to comprehend longer or more complex messages without intense concentration. A key reason for this "incomprehensibility" is the lack of proper prosody in synthetic speech. Prosody refers to the rhythmic and melodic characteristics of speech, which are used by the speaker to structure information for the listener. That is, prosody conveys to the listener which words or phrases are important prominence, and which words belong together in some semantic or syntactic sense (phrasing). Prosody involves a host of acoustic features, such as variations in fundamental frequency (F0), timing, and features that are related to the speaker's level of effort. Current synthesizers have poor prosody for two main reasons: (i) accurate prediction from text of timing and F0 is intrinsically difficult, and (ii) they can neither predict nor control features in speech that correspond to the speaker's articulatory effort. While many techniques exist for control of segmental duration (one aspect of timing) and F0 characteristics of speech, little attention has been paid to control of this second category of effects, and the quality of current synthesizers is poor as a result. The PI has defined a concept of "degree of articulation" to refer to the fact that, at a given speaking rate, speakers can control the precision and speed of the motions of their tongue, lips, velum, etc. with varying degrees of effort, from "hypo-articulate" (sloppy) to "hyper-articulate" (precise). Acoustic correlates of degree of articulation have been shown to covary with linguistic factors such as word emphasis and syllabic stress. While clearly important, this concept is nevertheless vague and its static and dynamic acoustic correlates have not been well established. Moreover, no quantitative models exist that predict degree of articulation from text or that provide a sufficiently precise quantitative description of these acoustic correlates for implementation in a synthesizer. The overarching goal of this project is to develop principled quantitative models for the prediction of acoustic features associated with degree-of-articulation, and to implement these results in a speech synthesizer. The strategy will be (a) to use text materials that systematically vary in prominence-related factors in order to elicit varying levels of degree of articulation in read speech; (b) to analyze speech signal, laryngograph signal, and jaw/lip articulatory data; and (c) to use the analysis results to generate mathematical descriptions of the relationship between prosodic structure and spectral features of the speech signal. The outcomes of this project will include the following: Improved understanding of the acoustic, glottal, and articulatory correlates of degree of articulation, including both static and dynamic features. This knowledge will impact not only basic science, but also technologies like speech synthesis and automatic speech recognition; Accurate prediction of spectral features of the speech signal from prosodic structure, based on a principled model that incorporates both acoustic and articulatory knowledge; Techniques for more natural-sounding speech synthesis that requires a lower attentional demand on the listener. This will lead to greater user acceptance of synthesized speech in applications including voice-based information access, language training, and tools for visually or vocally disabled persons.