Perception and Action: Ideal Observers and Actors

New York University, New York NY

Investigators

Linked publications & trials

Abstract

? DESCRIPTION (provided by applicant): Almost all aspects of decisions made or actions performed are typically stochastic. There are five components to the planning of an action based on sensory information. First, the subject has prior information about the state of the environment, which can be summarized as a probability distribution across possible world states reflecting knowledge of where things are and what scenes one is likely to encounter. Second, the subject has sensory input about the current state of the environment, which is uncertain due to physical and neural noise. Third, these two sources of information are combined to decide on an intended action (button press, arm or eye movement, or a complex plan that includes responses to potential subsequent sensory inputs). Fourth, the resulting action can differ from the intended one due to motor noise. Finally, the interaction of the resulting action with the current environment leads to a consequence (a loss or gain), and this consequence may be uncertain as well. As a result of all these stochastic components, visual tasks and movement planning require calculations that are equivalent to decision-making under risk. In our recent work, we have demonstrated that humans are often nearly optimal in visuomotor tasks in that they maximize expected gain, although we have found other circumstances in which human behavior is suboptimal. The tasks we perform every day are often new, never confronted before, or change from one attempt to the next, and thus require us to adapt to new circumstances. The aims of this proposal are unified by the study of learning in making decisions or actions under risk. We propose experiments to better understand the nature of human behavior in visual and visuomotor tasks, concentrating on how humans adapt to an uncertain and dynamic environment. We often use tasks with an experimenter-specified reward/penalty structure; this novel approach allows us to compare behavior with the optimal strategy that maximizes expected gain. We ask the following questions and propose experiments to address each. (1) How do observers select and adjust their decision criterion in tasks in which the statistics of the items to be discriminated are uncertain and possibly change over time? (2) When time is costly, how do observers decide when it is time to make a sensory decision, and when do they instead choose to gather more evidence before making that decision? (3) How do humans recalibrate and plan visually guided movement when those movements may be planned or may require recalibration in multiple encodings of the target of the movement? In all three aims we use patterns of performance (while performing a reach or making a perceptual decision) to learn about the underlying encoding of visual stimuli, uncertainty, and visually guided movement. These studies will shed light on the way in which visual stimuli and movements are encoded, perceptual decisions are made, and on how vision is used to guide action in an ever-changing world.