CompCog: Reading as a rational process of visual information gathering

Northwestern University, Evanston IL

Investigators

Abstract

Reading is a near-ubiquitous behavior in modern industrialized societies and is now one of the primary ways humans communicate. The goal of this project is to advance the scientific understanding of reading and the language processing that underlies it by rigorously testing a computational theory of reading. This theory models reading as an implicit process of moving the eyes to most effectively gather visual information for language processing. To accomplish this goal, a set of open-source computational models implementing this theory will be developed, implemented, and quantitatively evaluated against existing datasets of human reading behavior and new experiments. This work will enhance basic knowledge of reading and lead to more precise computational models of the reading process, which will enable the development of technologies that improve reading efficiency across the population. Additionally, reading is a highly complex learned skill, requiring tight coordination between vision, language processing, and motor control. Thus, the work will also advance knowledge of how the brain learns to solve complex problems and will contribute to scientific debates on the extent to which the brain learns maximally efficient solutions. The open-source computational models developed in this project will allow researchers who work on reading and language processing to formalize their predictions for reading behavior more precisely and to perform more sophisticated analyses of reading data. The investigator will train a diverse group of undergraduate and graduate students to conduct interdisciplinary research, requiring simultaneous proficiencies in computational modeling and human experimentation and knowledge spanning the disciplines of linguistics, computer science, and cognitive psychology. Finally, the results of this research will be incorporated into courses taught to broad audiences at summer schools, increasing the visibility and development of computational psycholinguistics as a field. The external manifestation of reading are eye movements, made as readers direct the high-acuity portion of the retina to subsequent points of interest in a text. Although eye movements in reading have an ultimate purpose of gathering visual information to identify the text, dominant models of eye movements in reading do not represent the process of text identification from visual information. By contrast, this project develops a computational model of eye movements in reading that does directly represent this process. Specifically, the theory formalizes reading as a process of probabilistically identifying the text being read by combining the reader's implicit statistical knowledge of the language with uncertain visual evidence. Eye movements are then sequentially guided rationally to obtain the most useful new visual evidence given the current state of text identification. To test the predictions of this account, the project will develop a set of open-source computational models instantiating the theory, each of which implements just a single aspect of reading behavior. Each model will incorporate state-of-the-art models of language knowledge and visual information and will make predictions for the reading of arbitrary texts. These advances enable this work to evaluate the theory robustly against human data and enable other researchers to extract predictions for their own data.