BCC-SBE: Seeing Speech: Building a Community

University Of Arizona, Tucson AZ

Investigators

Abstract

Human language integrates several complex systems. Sound is the most accessible of these, with measurable acoustic and articulatory properties, yet it is poorly understood because it involves multiple interacting modalities: lips, velum, tongue, and larynx all conspire to give each sound its unique properties. Understanding how sound works in language is important to basic linguistic and cognitive sciences as well as to applied research such as speech sciences and automatic speech recognition. Technological advances make collecting articulatory data relatively trivial, while technology for annotation and analysis lags behind. In response, we are developing (i) a software suite for simultaneous extraction and automatic analysis of articulatory speech data (UltraPraat), integrated seamlessly with the premier software for acoustic analysis of speech (Praat); and (ii), a database coupling articulatory and acoustic speech data (UltraSpeech) to support development and evaluation of theories of acoustic and articulatory phonetics, based on an existing database such as TIMIT. The success of such a venture depends on both the quality of execution and on how well it is received by the community. The goal of this project is to determine the key properties for both software and database that, if developed, would be most beneficial to the community of users. This project will create a prototype of the analytic software, develop the community of potential users, and refine software and database specifications to best meet the community's needs. The outcomes will be a working prototype, a community of researchers who understand its benefits, and a set of specifications for the infrastructure. This project brings together language researchers with a wide range of interests in human articulation. The discussions will ensure that the infrastructure developed to fill the current gap in articulatory data and analysis software will provide as much benefit as possible to all the related fields in technology and the sciences; they may also give rise to new research synergies. The prototype will be made available publicly, so others may begin using the results of this project for research, even before the full model is complete. UltraPraat and UltraSpeech have the potential for transforming research in all domains of oral tract articulation, such as the diversity of human language sounds, language acquisition and endangered languages, speech deficits, foreign language pronunciation, oral language for the profoundly deaf, speech recognition and synthesis software, and how musicians shape sounds while playing wind instruments.