The Sensorimotor Dynamics of Naturalistic Child-Parent Interaction and Word Learning

Indiana University, Bloomington IN

Investigators

Abstract

This award is funded under the American Recovery and Reinvestment Act of 2009 (Public Law 111-5). Children begin to comprehend words at 9 months. They say their first word at around 12 months. The pace of vocabulary learning then accelerates so that by 24 to 30 months, children add words at the staggering rate of 5 to 9 new words per day. There have been many studies focusing on documenting developmental progress in early language acquisition, and most theories of learning derived from those studies have focused on macro level descriptions that sound like explanations, such as "the mother tried to elicit the child's attention by waving the toy." These descriptions may capture higher-level human behaviors, but they fall short of a mechanistic account of how word learning works in real time. Toddlers learn words through millisecond by millisecond, second by second, and minute by minute events that are generated by actively engaging in the world, with objects, and with their social partners. But very little is known about how any of this works in real time and in the cluttered context of the real world interactions of toddlers and parents, contexts typically characterized by many interesting objects, shifts in attention by each participant, and goals (beyond teaching and learning words). In light of this, the series of experiments in this project will provide a systematic study of child-parent interaction and learning as coupled complex systems. The child's actions (head and eye movements, hand movements, picking up objects) create within the child dynamic dependencies of looking, seeing, touching and feeling. Each moment of perceptual and motor activity by the learner determines the next -- a head turn determines what is seen next, which may determine what is reached for and brought close to the eyes, which selects and generates the next view. Thus, the learner is a dynamic complex system. But the toddler is not alone when learning new words. Instead, a mature partner -- who is also a complex multimodal system -- offers words, gestures and actions. Critically, the streams of touches, sights and sounds from two participants are closely coupled, with one agent shaping the experiences and behaviors of the other. The study will measure the dynamic multimodal behavioral patterns within and across social partners as children and parents actively engage with and talk about objects in everyday contexts. The project will collect multiple streams of high-resolution, high-quality video and speech data from both participants. The dense and rich streams of multimodal data are useful only to the degree that one can find meaningful patterns in those dynamic streams that bring new insights into real-time learning events. To this end, the project will develop new methods of data analysis, visualization and data mining to quantify fine-grained behavioral patterns within an individual's cognitive, perceptual and motor systems and across social partners. This constitutes a significant advance in theoretical approaches to early word learning and one that also has broad applications. Measuring interaction patterns within and between complex systems is a critical problem across science -- from cells, to brains, to coupled physical systems, to human-computer interaction, to groups of animals, to teams of people. Thus, this research will bring new methods and analytic tools for measuring the information in coupled interactive systems. Understanding learning mechanisms in the context of a dynamic, everyday learning environment is essential to understanding typical development, individual differences, and atypical development. Designing effective procedures to benefit children with developmental delays requires a principled understanding of that dynamic environment as it relates to the cognitive learning system. Thus, the work will provide scientists, educators, and parents with an understanding of children's early cognitive processes and general principles to facilitate child-parent social interaction and early language learning. Moreover, building anthropomorphic machines that can acquire language automatically may be best accomplished by emulating how toddlers learn language. Artificial intelligence systems with human-like language skills have important utilities in real-world applications. Finally, this approach is methodologically novel. Not only will it provide new findings, but the research will be a proving ground for the development and invention of these new techniques -- techniques that may be applied in many different domains of social and behavioral studies, such as typical and atypical cognitive development, collaboration and joint problem solving, and adult social interactions.