RI: Small: Computational analysis of eye movements in reading: reader characteristics, cognitive state, and natural language processing

Massachusetts Institute Of Technology, Cambridge MA

Investigators

Abstract

Reading is the most widely practiced skill in the world. It occupies many hours of our daily lives and is crucial to functioning successfully in modern society. When we read, our eyes move over the text in a way that reflects how we perceive, process, and understand it. This project uses eye tracking to develop a new approach to studying how our eyes move during reading and what eye movement patterns can reveal about readers' linguistic knowledge and how they interact with text. In particular, the researchers develop eye tracking-based methods that automatically determine readers linguistic proficiency, how well they understand the text, and how difficult they find it. The project improves automatic text processing by machines, by taking into account information about how humans read. This research program benefits society by advancing our scientific understanding of language processing during reading and enabling new technologies that support human readers from a wide range of backgrounds and skill levels. It lays the foundations for future digital platforms that make it easier for people to access textual information, improve literacy, learn new languages, and personalize text content and complexity according to readers' needs and goals. The project introduces a novel conceptual and computational approach to studying human reading with both native and non-native speakers, by leveraging broad coverage eye movement patterns during reading of free-form text. It develops a computational framework that connects eye movement in reading to linguistic properties of the text, the readers' linguistic knowledge and their cognitive state during reading. To realize this framework, the project first focuses on using eye movement in reading to automatically predict readers' linguistic proficiency and estimate their comprehension of specific parts of a text. These and other related tasks help to characterize how language comprehension manifests in gaze and unfolds over time, and also lead to the development of a predictive computational framework for native and non-native reading. Its second focus is on integrating data and representations from eye tracking in natural language processing, with the aim of developing applications which support and enhance human reading and language learning. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.