INSPIRE: The RAVE Revolution for Children with Minimal Language Experience During Sensitive Periods of Brain and Language Development

Gallaudet University, Washington DC

Investigators

Abstract

This Integrated NSF Support Promoting Interdisciplinary Research and Education (INSPIRE) project "The RAVE Revolution for Children with Minimal Language Experience During Sensitive Periods of Brain and Language Development" is jointly funded by Robust Intelligence and Cyber-Human Systems Programs of the Computer and Information Science and Engineering Directorate; the Developmental and Learning Sciences Program, and the Science of Learning Centers Program of the Directorate for Social, Behavioral & Economic Sciences; the Office of International Science and Engineering; and the Office of Integrated Activities. The interdisciplinary team expands the boundaries of traditionally separate disciplines by uniting synergistically to explore a transformative learning tool to reduce the devastating impact of minimal language experience on children. Children's dramatically reduced language experience has been shown to have a deleterious impact on learning language, reading, and achieving normal cognitive functions across life,a problem facing many children in the nation (e.g., children from low SES backgrounds; late-exposed bilingual children). Deaf babies are at particular risk, as beyond minimal language experience, many receive no accessible language experience in early life. The Robot AVatar thermal-Enhanced learning tool, or RAVE, is placed near a baby's high-chair and makes available multiple core components of sign language, with speech options, in first-time socially interactive ways, and, crucially, during critical periods of human brain and behavior development. Exploration of this revolutionary learning tool involves scientists spanning four disciplines (Developmental Cognitive Neuroscience, Virtual Human Science, Robotics, and Applied Psychophysiology/Biomedical Engineering), provides science answers about how all babies discover core components of language, resolves previously insoluble problems in the four sciences, and is propelled by the shared objective to enhance early learning gains for populations that would otherwise be at a lifelong disadvantage. Of particular novelty, fNIRS brain imaging provides first-time discovery of babies' sensitivities to multiple rhythmic components underlying language vital to healthy language learning. This forms the basis for creating rhythmic patterns at the core of nursery rhymes and conversational samples in sign language (through Motion Capture) and speech. Thermal IR imaging+eye tracking identifies when deaf babies who cannot yet produce language are in a peaked arousal state and ready to learn. This triggers the robot+virtual human to start, cease, or solicit interaction based on the baby's social engagement. When the baby's gaze locks with the robot, the robot initiates gaze direction to virtual humans, initiating joint attention and socially-contingent conversation in human artificial agent interaction. This triggers an interfaced screen where virtual humans provide rhythmic nursery rhymes and conversations in sign language, with speech options. Together, the RAVE team explores a new aid to children with minimal or no language input; provides the nation with a competitive science and technological edge; and, trains under-represented groups in STEM, students in interdisciplinary science, and young deaf scientists in the advancement of science with transformative translational significance for all society.