Vision in Natural Tasks

University Of Texas At Austin, Austin TX

Investigators

Linked publications & trials

Abstract

DESCRIPTION (provided by applicant): The proposed research will attempt to provide a unified theory of task control of gaze and walking trajectories as humans move through natural environments. Until recently this goal would have been intractable, but a number of recent research results have illuminated the connection between simple sensory- motor decisions and behavioral goals. In particular, reinforcement-learning algorithms use reward signals to predict optimal behavior, and the central role of reward is well established in neurophysiological studies. Nonetheless it is unclear how these mechanisms determine natural visually guided behavior. Since natural gaze behavior is tightly linked to behavioral goals, reinforcement learning has the potential for understanding how behaviorally relevant targets are selected. We will develop a theoretical framework based on reinforcement learning for understanding sensory-motor decisions when humans move through natural environments. We first use Inverse Reinforcement Learning methods to estimate the internal reward associated with different behavioral goals when subjects navigate through obstacles and targets in a virtual environment, and then use the estimated reward values to predict the specific fixation sequences made while performing the task. We will test whether reward-weighted uncertainty determines gaze changes, predict gaze allocation in novel environments, and test how reward and uncertainty combine. A critical feature of the approach taken here is the decomposition of complex behavior into a set of sub-tasks. This approach has the potential for making complex behavior theoretically tractable and we will test this assumption. We will attempt to identify and quantify the potential sources of uncertainty such as sensory encoding, decay in spatial working memory, and uncertainty stemming from the observer's own motion in the environment Prior knowledge of an environment allows more efficient allocation of attention to novel or unstable regions. We will attempt to model the development of memory representations as a reduction in uncertainty, and evaluate how prior knowledge changes attentional allocation in uncertain environments. The work represents a major advance by developing a theoretical context for understanding selection of gaze targets in a moving observer. To date, formal theoretical approaches to decision making have addressed highly simplified scenarios. Because we are investigating natural vision there are very direct implications for both clinical and human factors situations involving multi-tasking. Eye movements are diagnostic of a variety of neural disorders and the exploration of normal gaze patterns in natural tasks provides essential data for comparison with disease states.