CRCNS US-French Research Proposal: Bayesian Models of Sensory Integration, Adaptation and Calibration

New York University, New York NY

Investigators

Abstract

A fundamental question in perceptual and cognitive science concerns how decisions about incoming sensory information unfold over time. The influx of available sensory information must be balanced across time with the need for quickness versus accuracy in making a decision. Sensory systems are flexible in response to changing context, resulting in several dynamic aspects of perception at multiple time scales, including the trade-off between speed and accuracy in decisions or actions, adaptation for repeated sensory stimuli, and long-term recalibration in response to a consistent change in stimulation or error. This proposal attempts to advance theoretical understanding of these processes by unifying disparate threads in modern perceptual theory while developing models of the dynamics of sensory integration and decision-making. The research could ultimately have implications for the design of virtual reality systems, lighting and sound systems, visual displays, and artificial vision and sound processing systems. A large literature on multiple cue integration, or how observers integrate multiple sources of information for making a perceptual decision, suggests that human behavior compares favorably with the predictions of optimal Bayesian models for combining multiple sources of information (including prior knowledge). However, this literature examines decisions under static conditions. There is a mostly distinct literature on speeded decision-making that instead asks how information is accumulated over time, leading to the combined decision of both what response to make and when to make that response. The novelty of the proposed research is to unify these two threads of perceptual theory by developing and testing a model that incorporates both a dynamic decision-making model for evidence accumulation (a diffusion process) and multiple-cue integration. Each cue contributes independently to the dynamic process of evidence accumulation. The experimental work proposed to test the model covers a wide range of sensory decision-making phenomena and thus is expected to have a strong impact on the field of perceptual science. However, the wider impact will result from the unifying models to be developed, as the combination of optimal cue integration with dynamic decision-making models has extremely wide applicability in cognitive science generally. A companion project is being funded by the French National Research Agency (ANR).