Vision in Natural Tasks

University Of Texas At Austin, Austin TX

Investigators

Linked publications & trials

Abstract

PROJECT SUMMARY/ABSTRACT The goal of this proposal is to understand human vision in the context of natural behavior. This is of fundamental importance because relatively little is known about how vision functions in the natural world, and many important issues arise in this setting that are absent, or difficult to address, in standard paradigms. It is our contention that understanding many aspects of vision, such as selective attention and control of gaze will be impossible without investigation of vision in its natural context. Previous attempts to explain gaze patterns have almost exclusively concerned only static, restricted stimulus conditions, and focused on the properties of the stimulus. Such models cannot extend to natural behavior where the visual input is dynamic, and where the observer's behavioral goals play a dominant role. The pervasive effect of reward in the neural circuitry underlying saccadic eye movements, the development of the mathematics of Reinforcement Learning, and the application of statistical decision theory to understanding sensory-motor behavior all allow a novel framework for understanding the sequential acquisition of visual information in the context of normal behavior. We explore evidence for this framework in the proposal. Specifically, we examine the role of reward, uncertainty, and prior knowledge in the control of gaze in the natural world, with the goal of providing a formal structure for understanding complex behavioral sequences. The research focuses on dynamic environments where a central open question is how the visual system balances the need to attend to existing goals against the need to maintaining sensitivity to new information that may pose opportunities or threats. We investigate the role of reward, uncertainty, and prior knowledge in control of gaze in experiments in an immersive virtual walking environment. We then test the predictions of a model based on reinforcement learning in a simple divided attention task. We then use recently developed methods of Inverse Reinforcement Learning to estimate the intrinsic rewards of different behavioral goals in the walking environment. This will allow us, for the first time, to infer intrinsic human rewards directly, and to predict gaze sequences in novel situations. We also explore the nature and complexity of prior knowledge and the role of prediction in intercepting moving objects in a virtual environment. Although investigation of natural behavior is challenging, the potential benefit is that it provides the empirical basis and theoretical tools for understanding how complex behavioral sequences are generated. The experiments will lay the groundwork for investigation of clinical populations in contexts such as walking, and will also have direct relevance to safety issues in driving and any situation involving multi-tasking.