Transsaccadic Memory and Scene Representation

Michigan State University, East Lansing MI

Investigators

Abstract

We experience the visual world subjectively as a full-color panorama of visual detail. This experience naturally leads to the belief that the human visual system generates a complete and truthful internal copy of the outside scene, similar to a detailed color photograph. Consistent with this intuition, past research has demonstrated that human visual memory for scenes can be exceptionally good. At the same time, it is well known that visual detail and rich color are only available where the eyes are directly pointed. To compensate for this constraint, our eyes flit from place to place over a scene in a series of very fast eye movements called saccades. Interspersed among these saccades are brief pauses, called fixations, and it is only during these fixational pauses that visual information is actually acquired from the scene. Therefore, if our visual system does in fact create a complete internal representation of the external world, as experience suggests, then this representation must be stitched together from the individual snapshots taken during each fixation. In contrast to this intuitively appealing view, there is a good deal of recent evidence that the human visual system does not construct such a high-fidelity copy of the world. For example, a remarkable recent discovery is that human viewers are often very insensitive to dramatic changes in the visual world that take place from one moment to the next. This finding suggests that despite experience and intuition, a photographic image of the entire scene is not concurrently available for comparison to the current state of the world. What, then, is the nature of the internal representation that is generated and retained over time by the human visual system? The main objective of this research is to understand the visual representations that arise as the human viewer examines the world dynamically over extended time. The research will be directed toward discovering the principles that underlie human visual perception, visual cognition, and visual memory. The research will use sophisticated methods that combine fast and powerful graphics manipulation and presentation systems with a highly accurate eyetracking system. Using these instruments, complex scenes will be changed in real time contingent on a specific eye movement within the scene, and the sensitivity of the visual system to such changes will be measured under a variety of conditions. The results of this work will expand our understanding of how the human brain gives rise to perceptual experience and visually guided performance and will help guide the design of new human-computer interfaces. The results will also help guide scientists in building the next generation of artificial vision systems.