Computational Neuroscience of Language Processing in the Human Brain

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Linked publications & trials

Abstract

The neural architecture of language is the foundation for the highest form of human interaction. Prior work has delineated a network of frontal and temporal brain areas that selectively support language processing, but the precise computations that underlie our ability to extract meaning from sequences of words have remained out of reach. Recent breakthroughs in natural language processing and machine learning have led to the development or artificial neural network (ANN) models of language that perfrom remarkalbly well on previously intractable tasks, providing the first computationally precise models of how these functions might be solved by the brain. However, the standard approaches in human cognitive neuroscience lack the spatial and temporal resolution necessary for precise comparisons to computational models. Here we propose to collect large sets of neural responses to language stimuli from the human brain using the only method up to the task-human intracranial recording, and to bring the recent advances in computational neuroscience and machine learning to bear on the ultimate scientific quest of understanding uniquely human linguistic ability. In Aim 1, we build on a strong foundation of our earlier fMRI work to tackle a fundamental distinction between two key components of language-understanding word meanings (lexico-semantic processing) and connecting those meanings into phrases and sentences representations (syntactic processing). This distinction will be tested across four controlled paradigms. In Aim 2, we will extend the results from Aim 1 to rich naturalistic materials, testing for sensitivity to word meanings vs. hierarchical syntactic structure in predicting upcoming words. We will also supplement this hypothesis-driven approach with a data-driven search for structure in neural responses to language using innovative analytic techniques recently developed in the study of auditory cortex. Finally, in Aim 3, we will use human intracranial responses to language stimuli to test computational models of language understanding. In recent work, we tested a large number of ANN language models and found that the most powerful 'transformer' models accurately predict neural responses. Here we harness the precision of human intracranial recordings, a large set of diverse linguistic materials-selected and created to discriminate among candidate ANNs using novel state-of-the-art design optimization approaches-and controlled minimal-pair network comparisons to powerfully test computational models of language understanding. In line with current emphasis in the field on robust, replicable, and open science, the full dataset (1,500+ sentences in each of ~40 participants) will be made public, enabling many analyses beyond the ones proposed here, thus turbo charging the quest to understand the neural computations underlying language processing and moving the field forward for years to come. This work will set the stage for the most comprehensive picture of the neural architecture of language and synergize with parallel ongoing advances in network modeling of human cognition, making it possible for the first time to discover the neural codes and computations underlying human language.