Examining syntactic and semantic computations when no order is imposed from the input

New York University, New York NY

Investigators

Abstract

Language can be conveyed through various modalities including sound, sight, and touch. Each of these modalities has its own timing characteristics. When studying the neurobiology of language, researchers often choose a specific modality, which hinders the ability to distinguish between language properties and modality-specific dynamics. Understanding how the neural pathways of language interact with different modalities is crucial for developing comprehensive models of language in the brain. This project examines how the language system organizes itself when the input lacks temporal sequencing because this should be a useful window into the brain’s inherent way to travel from form to meaning. The internal timing aspect of sentences is removed by using short written stimuli, flashed rapidly all at once, similarly to quick notifications on a phone. Prior research has shown that this type of flashing stimulus can be understood quickly, but only if the stimulus is a grammatical sentence. This project uses noninvasive brain measurements to reveal the spatiotemporal dynamics of syntactic and semantic computations for a stimulus that contains no internal timing dynamics of its own. In addition to advancing the neurobiology of language, this project sheds light on how human brains swiftly grasp meaning from a single glance – an ability critical in today's visually saturated world. The project involves mentorship of a diverse set of undergraduate students, graduate students, and a post-baccalaureate research assistant. Two high school students are integrated into the project as lead investigators of their own studies. Multiple languages are examined, and some studies include outreach to relevant speaker communities. In this project, magnetoencephalography (MEG) and electroencephalography (EEG) are used to examine two fundamental questions: (1) what is the inherent spatiotemporal organization of syntactic and semantic computations in the brain, and (2) how does the brain map input to linguistic knowledge in the absence of predictions from a temporally preceding context? Each study elicits a neural Sentence Superiority Effect (SSE), manifested as increased neural activity for sentences as compared to unstructured stimuli. The underlying function of each neural SSE is investigated by introducing errors and representational complexity to the stimuli to determine what “counts” as a sentence for those neural signals. The degree of serial processing within each neural SSE is also assessed by examining the effects of transition probabilities between words in a left-to-right fashion across the entire stimulus. If language processing is inherently serial, then serial processing should manifest itself even for a parallel stimulus. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.