STREAMLInED: Shared Tasks for Rapid, Efficient Analysis of Many Languages in Emerging Documentation

University Of Washington, Seattle WA

Investigators

Abstract

This project aligns the research interests of two separate scientific and engineering communities in order to push the boundaries of automatic speech processing technology and bring its benefits to the urgent task of endangered language documentation. Automatic speech processing technology has become familiar in the everyday lives of many speakers of English and other widely spoken languages through tools such as automatic captioning and voice-driven personal assistants. Meanwhile, linguists are rushing to document and analyze the thousands of languages that by the end of this century, will no longer be acquired by children. Such work would be greatly assisted by automatic processing of recorded spoken endangered language data. Modern automatic speech processing tools, however, require training data sets orders of magnitude larger than what is available for endangered languages. This project will advance scientific knowledge on this problem by structuring a "shared task evaluation challenge" around language documentation-based data sets. Better language documentation puts communities in a better position to undertake language revitalization, which in turn can be a key component of community development for marginalized populations. Broader impacts also include the benefits of bringing speech technology that works with small data sets to widely spoken but understudied languages, often languages of communication in regions of geopolitical and economic importance to national interests. Language documentation projects typically begin with large quantities of recorded speech. Turning that spoken signal into a transcribed form is a major bottleneck in the language documentation process. Similarly, language archives house recorded, unanalyzed data from many languages with no living fluent speaker, but which have communities interested in revitalizing their heritage languages. At the same time, the development of technology that can work effectively with very small training data sets is an open and interesting challenge for speech researchers. The shared task evaluation challenge framework provides the structure of a friendly competition in which different research groups can explore and compare approaches that are evaluated with standardized data and metrics. This strategy for focusing research effort has advanced the frontiers of language technology for decades. This project will apply it for the first time to the specific challenges of endangered language documentation: working with truly low-resource languages, with often noisy or other imperfect recording conditions. The specific tasks the challenge will focus on include: identifying the language and speaker of each segment of a recording, identifying the genre (e.g. story telling vs. dialogue) of segments of recordings, and aligning short partial transcriptions to the spoken recordings. The researchers will prepare the data (based on existing data sets identified in language archives), set up functioning baseline systems that task participants can use for comparison and/or build on further, establish evaluation metrics, and execute the shared task. The shared task structure will encourage and support participants in making their contributions open source, with an eye towards ensuring they are available to language documentation researchers. The project will also include outreach to the language documentation community in order to train such researchers in the use of the technology developed. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.