Collaborative Research: Multimodal State Estimation through Neural Coherence in the Parieto-Frontal Network

Arizona State University, Scottsdale AZ

Investigators

Abstract

To distinguish parts of our body from other objects around us, the brain needs to build an internal image of our body by merging information from the skin, muscles and joints with information from our eyes. This project is aimed at characterizing how we build our sense of self, by using multisite brain recordings and virtual reality technologies. The results will impact national needs in the consumer, healthcare, military, and industrial settings by advancing the fundamental engineering and neuroscience knowledge necessary to create the next generation of brain-machine interfaces, which are envisioned to support the integration of artificial and natural sensory information. Optimizing these systems depends critically on understanding how natural sensory signals interact within and among brain areas, a knowledge gap that directly addressed by this project. The educational goals associated with the project are designed to advance discovery and understanding of engineering and neuroscience, while also promoting teaching, training, and learning beyond the regular bounds of these disciplines. These goals are achieved by: 1) Engaging high school students underrepresented in STEM fields through the development of hands-on instructional modules, 2) Promoting interdisciplinary undergraduate research opportunities via internships at Arizona State University, 3) Mentoring students in the broader implications of scientific research through exposure to organizations engaged in the ethical, societal, and policy implications of neuroscience research, and 4) Engaging the public in scientific discourse through public lectures and exhibits, and thereby promoting broad dissemination of the work to enhance scientific and technological understanding. Estimating the state of the body through the integration of available sensory cues (multimodal state estimation) is a critical integrative function for most organisms. Although much is known about state estimation for the upper limb at the behavioral level, the underlying neural mechanisms remain poorly understood in cortical areas, particularly at the network level. This is due to several factors: 1) the cortical areas believed to play a role in limb state estimation are heterogenous with regard to the relative strength of their sensory inputs and display both multisensory enhancement and suppression depending on context; 2) technical limitations mean functional interactions among these areas have been challenging to characterize; 3) the relation between sensitivity to visual and somatic cues and prevailing computational theories of multisensory integration have been incompletely explored; 4) multimodal areas are thought to contribute to both perceptual and action-based body representations but how these representations interact at the neural and behavioral levels is not well understood. As a result, it is unclear how a coherent multimodal estimate of the state of the upper limb is constructed and maintained. The proposed series of studies address these issues by quantifying changes in neural spiking, local field potentials, and neural coherence within and among fronto-parietal areas of the monkey implicated in state estimation using virtual reaching tasks that alter the reliability and semantic information of visual cues.