RIDIR: Collaborative Research: Enabling Access to and Analysis of Shared Daylong Child and Family Audio Data

University Of California-Los Angeles, Los Angeles CA

Investigators

Abstract

A child's language development in the first few years of life predicts long-term cognitive development, academic achievement, and expected income as an adult. Early language development in turn depends on linguistic interactions with adults. Increasingly, researchers are using daylong audio recordings to study child language development and child-caregiver interactions. Compared to short language samples, daylong recordings capture the full range of experiences a child has over the course of a day. Daylong audio recordings are also being used in applied settings. For example, studies show that by the time they enter First Grade, children from higher socioeconomic backgrounds hear tens of millions more words than children from lower socioeconomic backgrounds, perpetuating social inequalities. Multiple large-scale intervention projects targeting low socioeconomic households, including the Thirty Million Words Initiative in Chicago and the Providence Talks program, are using daylong audio recordings to provide automated, personalized feedback to parents on when and how often their child hears adult words and experiences conversational turns. The features of daylong recordings that are advantageous for researchers and practitioners also pose unique challenges. For one, their long durations are ideal for studying the temporal dynamics of child-adult interaction, but taking advantage of the long durations requires the enlistment of automated speech recognition technology. Current automatic speech recognition systems have difficulties with child speech and are challenged by the noisy and varied acoustic environments represented in the recordings. Another challenge is that the recordings capture private moments that require long hours of human listening to remove. This makes it difficult for researchers to share the recordings publicly, so that the potential value of the recordings collected by individual research labs is not fully realized. This project will create a new resource, called HomeBank, that will have three key components: (1) a public dataset containing daylong audio recordings that have had private information removed by human listeners, (2) a larger dataset containing about ten to one hundred times as many hours of recording that have not had private information removed and will be free but restricted to those who have demonstrated training in human research ethics, and (3) an open-source repository of computer programs to automatically analyze the daylong audio recordings. HomeBank will take advantage of an existing cyberinfrastructure for sharing linguistic data called TalkBank. The daylong audio recordings included in the datasets will represent both typically developing and clinical groups, a range of ages from newborn infants to school age children, and a range of language and socioeconomic backgrounds. We expect the primary users to be basic and applied child development researchers as well as engineers developing automatic speech recognition technologies. The free-to-access database and the open source computer programs will ultimately improve both the data on which early interventions are based and the tools available for providing parents with feedback on the linguistic input they provide their children. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.